Acceptable Use Policy (AUP)
1. Permitted Uses
FlyCrawl is designed as a neutral technical conduit for ingesting publicly accessible web data into modern computational pipelines. Permitted uses include:
- Retrieval-Augmented Generation (RAG) and LLM context enrichment from public documentation.
- Academic and scientific research on public datasets.
- Search engine indexing, price monitoring, and competitive intelligence of public pages.
- Developer testing, SEO auditing, and web performance benchmarking.
2. Prohibited Activities
You agree NOT to use FlyCrawl for any of the following strictly prohibited activities:
- Denial of Service (DoS): Launching coordinated high-frequency requests with the intent or effect of crashing, overloading, or disrupting target websites.
- Personal Data Harvesing (PII): Scraping personally identifiable information, healthcare records, credit card numbers, or passwords in violation of privacy laws.
- Bypassing Authentication Barriers: Extracting private data located behind login systems or paywalls without legitimate account rights or permissions.
- Malware, Phishing & Fraud: Any scraping supporting credential harvesting, scamming, or illegal operations.
3. Target Server Respect & Legal Responsibility
Users remain solely responsible for ensuring their data scraping activities comply with target site terms of service, robots.txt directives, and governing jurisdictions (including CFAA and GDPR). FlyCrawl provides rate limiting controls and urges all developers to configure concurrency responsibly.
4. Domain Exclusion & Abuse Reporting
If you are a webmaster or site owner and wish to exclude your domain from all FlyCrawl crawlers, please contact our abuse desk or submit our automated exclusion request: