The Ethical Web Data Collection Initiative (EWDCI) is committed to promoting ethical, transparent, and legally sound web scraping practices. As web data aggregation grows in importance, it’s crucial for businesses and developers to understand the legal nuances that surround web scraping, particularly the role of terms of service (ToS) agreements.
Understanding Terms of Service
Web scraping is governed by multiple legal frameworks, including contract law, which comes into play when scraping websites with ToS agreements. These agreements can vary significantly from one website to another, and understanding their impact on web scraping is critical to ensuring compliance with legal and ethical standards.
There are two primary types of ToS agreements that scrapers need to be aware of:
1. Clickwrap ToS: These require explicit agreement by the user, typically through a button click or checkbox confirming acceptance. For example, when signing up for a user account or accessing certain features, you might be required to actively agree to the terms before proceeding. When scrapers encounter Clickwrap ToS, these create a binding contract—meaning the scraper must fully comply with the terms, including any prohibitions on scraping. The EWDCI advises strict adherence to these agreements to avoid potential legal disputes.
2. Browsewrap ToS: Unlike clickwrap, browsewrap agreements are passive and do not require explicit consent from the user—nor do these terms even draw attention to themselves. These terms are typically embedded somewhere on the website, often accessible through a link at the bottom of the page. The EWDCI holds the position that Browsewrap ToS do not always form binding contracts as users are not necessarily on notice of these terms, nor do they take any active steps to accept them. In cases where only browsewrap terms exist on public websites, the EWDCI believes that web scrapers may proceed with collecting publicly available data, as long as they remain in compliance with all other applicable legal and ethical principles.
Scraping and Responsible Use of Public Data
From multinational efforts like GDPR and the EU AI Act to single-state and -jurisdiction laws such as California’s Consumer Privacy Act, many different—and sometimes overlapping—legislative efforts will impact a company’s web scraping strategy. Beyond the letter of the law, there’s great value in establishing an industry-wide ethical compass, both to build trust with the public and preempt the need for legislation that may not be fit for purpose. Building that trust is a proactive process that no one company can do alone.
As such, while the legal enforceability of browsewrap ToS remains a topic of debate, the EWDCI urges the industry to prioritize ethical data practices. Regardless of the ToS structure, web scraping should always be conducted in a way that respects user privacy, avoids circumventing technical safeguards that prevent access to non-public data, and minimizes any potential impact on website performance. Public trust in web data collection can only be maintained when businesses act responsibly, even in legally gray areas.
The EWDCI’s Ethical Standards
The EWDCI is dedicated to fostering transparency and accountability in web data aggregation. Our guiding principles emphasize:
• Legality: Compliance with local and international laws is essential, including respect for binding ToS agreements.
• Ethical Responsibility: Web scraping should always be conducted in a way that protects individual privacy and adheres to the highest standards of consent and transparency.
• Innovation: By promoting responsible scraping, we aim to unlock the full potential of data aggregation for industries like e-commerce, research, and public services, while avoiding practices that could harm public trust.
By addressing these challenges, the EWDCI seeks to create an environment where web data collection remains a trusted and indispensable tool, governed by clear ethical and legal standards.
About EWDCI + i2Coalition
The Ethical Web Data Collection Initiative (EWDCI) seeks to foster cooperation in the web data collection and aggregation industry and leverage collective first-hand knowledge and insights to advocate for beneficial technical standards and business best practices regarding the aggregation of data. The EWDCI is dedicated to serving as the voice of the industry, collaboratively strengthening public trust in the practice of Data Aggregation, promoting ethical guidelines, and helping businesses make informed data aggregation choices.
The Internet Infrastructure Coalition (i2Coalition) is the leading voice for web hosting companies, data centers, domain registrars and registries, cloud infrastructure providers, managed services providers, and related tech. The i2C works with Internet infrastructure providers to advocate for sensible policies, design and reinforce best practices, help create industry standards, and build awareness of how the Internet works. The i2Coalition also spearheaded the creation of the VPN Trust Initiative, which determined and promoted best practices for that vital industry.