At two in the morning, a business analyst at a retail company runs a script he wrote a week earlier. The script pulls prices of thousands of products from a competitor's website and stores them in a database. He smiles as the data streams in: 12,400 prices in one night, enough to build next month's pricing strategy. Six months later, his company receives a legal demand letter from the competitor, followed by a police report alleging violations of the ITE Law and unauthorized access. The analyst's question is simple: "But the data is public, right?"
That question is the same one thousands of Indonesian businesspeople ask every year. Web scraping — automatically extracting data from websites — feels like a gray area: the data looks public, but the way you obtain it can break the law. And in Indonesia, the legal answer is not as simple as "public means free."
This article covers the legal and ethical aspects of web scraping for business in Indonesia: what the laws actually regulate, what counts as a violation, what ethical practice looks like, and how to get the data you need without building a legal time bomb.
What Is Web Scraping, and Why Businesses Use It
Web scraping is the process of extracting data from websites automatically using programs, rather than copying it by hand one by one. It can be as simple as a script that downloads a price list, or as complex as a system that monitors thousands of sites every hour.
Common legitimate business uses include:
- Competitor price monitoring: retailers track competitor pricing to build strategy.
- Market research: collecting product reviews, trends, and consumer sentiment from forums and marketplaces.
- Collecting public data for research: statistics, weather, or publicly published government data.
- Content aggregation: portals that gather news or listings from multiple sources.
- AI model training: collecting large volumes of text data to train language models.
On the other side, scraping is also used for problematic purposes: stealing content for republication, collecting people's personal data without consent, overwhelming sites with excessive requests, or stealing protected commercial databases.
The critical point is not the scraping technology itself, but three things: what data is collected, how it is collected, and what it is used for.
The Legal Framework in Indonesia: What Is Actually Regulated
It is important to understand: Indonesia has no single law called the "Web Scraping Act." These cases are adjudicated through a combination of several laws. Here are the most relevant ones.
The ITE Law (Law No. 11 of 2008, amended by Law No. 1 of 2024)
The Electronic Information and Transactions (ITE) Law is the main umbrella. Two articles are most often used in cases involving access to electronic systems:
- Article 30: prohibits unauthorized access to another person's electronic system, by any means. Violations carry up to 8 years' imprisonment and/or fines up to Rp 12 billion.
- Article 31: prohibits interception or recording of another person's electronic information transmissions without authorization.
The big question: is accessing a public website considered "unauthorized"? This is where interpretation matters. Accessing pages that are genuinely open to anyone is generally viewed differently from breaking through login or circumventing technical protections. However, if a site explicitly prohibits scraping in its terms of service, or if scraping is done by bypassing protections (such as circumventing CAPTCHA or protected APIs), the criminal risk rises sharply. Courts in different countries also disagree; some hold that scraping public data is not unlawful, while others punish it as a breach of terms of service.
The Personal Data Protection Law (Law No. 27 of 2022)
This is the law that has changed the landscape most for scrapers in Indonesia. The PDP Law protects personal data — information that can identify a person, such as name, phone number, address, email, health data, and personal preferences.
The implication is immediate: collecting people's personal data through scraping, without a lawful basis for processing, violates the PDP Law. Lawful bases include data subject consent, contractual obligation, or a balanced legitimate interest — but "legitimate interest" cannot be used casually for personal data scraped in bulk from social media or marketplaces.
The sanctions are real: unlawful collection of personal data can lead to up to 5 years' imprisonment and fines up to Rp 50 billion for corporations. This is no longer a gray area; it is written clearly in the statute.
The Copyright Law (Law No. 28 of 2014)
Copyrighted content — articles, photos, videos, designs — cannot be copied and republished without permission, regardless of method. Scraping does not change this status. If you take news articles to paste onto your own site, you are violating copyright, period. Legitimate aggregators use official APIs or licensing agreements with publishers.
Monopoly and business competition rules
Systematically collecting competitor data to imitate them can run into Law No. 5 of 1999 on the Prohibition of Monopolistic Practices and Unfair Business Competition, especially where the result is unfair practices or market disruption. Such cases are rarer, but they are serious considerations for large market players.
Civil contracts: Terms of Service
Do not forget the layer that surprises scrapers most: a site's terms of service (ToS). Many sites explicitly prohibit scraping in their ToS — including major marketplaces in Indonesia. Violating ToS is a breach of contract, regardless of any criminal liability. Breach of contract can ground a civil suit with substantial damages, and in some jurisdictions becomes supporting evidence for criminal charges. If you scrape using an account — for example, a marketplace account registered in your name — the trail leads straight back to you.
We are not a law firm and this article is not legal advice. For concrete cases, consult an advocate who understands Indonesian cyber law before building a large scraping system.
Scraping Ethics: The Unwritten Rules That Still Decide Outcomes
Law is the minimum; ethics is the recommended boundary. Ethical scraping practice keeps you away from conflict, bad reputation, and shifting legal direction. Four basic principles:
1. Respect robots.txt and opt-out mechanisms
Robots.txt is a small file on every site stating which parts bots may and may not access. If a site writes "Disallow: /search", respecting it is basic ethics. It is not directly binding law, but ignoring it signals bad faith in the eyes of courts.
2. Do not overload servers
Scraping thousands of pages per minute can damage another site's servers, slow its service for real users, and potentially be categorized as an attack (denial of service). Polite scraping runs slowly, with pauses between requests, outside peak hours. If a site asks you to stop — through blocking or an official message — stop.
3. Separate personal data from general data
Product prices, anonymous reviews, and public statistics differ from names, phone numbers, and addresses. If your needs touch personal data, stop and find a lawful path: official APIs, cooperation with the data owner, or licensed data services. This principle aligns with the PDP Law's strict regulation of personal data collection.
4. Do not republish protected content
Taking data for internal analysis differs from displaying it again on your site. If an analogy helps: quoting sparingly with attribution differs from copying an entire article. For content aggregation, use official APIs or licensing agreements.
5. Internal transparency
Your team needs to know that scraping systems are running and what their limits are. "Quiet" scraping by a single employee is the biggest risk: a company can be dragged into a problem no one ever approved. Document what is collected, from where, and for what purpose, so legal and ethical decisions are made deliberately.
Safer, Lawful Alternatives: Getting Data Without Shooting Yourself in the Foot
The good news: almost every legitimate business data need has a safer path than rogue scraping. Here is the order, from safest down.
1. Official APIs
Many platforms provide official Application Programming Interfaces (APIs) for extracting data legally and in structured form: marketplaces, financial services, government, media. Official APIs usually have quotas, documentation, and sometimes fees — but they are legal, stable, and carry no legal risk. Examples: statistics through government agency APIs, prices through official data provider APIs, or content through platform APIs that offer them.
2. Open public data
The Indonesian government and many institutions release open data intended for the public: BPS statistics, geospatial data, trade data, and more. Using open data removes almost all legal risk, because the data owner explicitly permits its use.
3. Cooperation and data licensing
For commercially valuable data — competitor prices, review data, market data — the healthiest path is buying from licensed data providers or making agreements with data owners. There is a cost, but the legal gap is closed. At scale, this is almost always cheaper than lawyers' fees and criminal risk.
4. Compliant third-party services
Some companies specifically sell scraped data with clear licensing. Use them when available; they have already borne the compliance risk on your behalf. Make sure the contract states the data's origin and compliance guarantees.
5. Limited scraping of clearly public, non-sensitive data
If scraping remains the only option — for example, monitoring a few price pages from a few sites for internal analysis — do it with strict limits: small volume, pauses between requests, respect robots.txt, no personal data, no bypassing protections, and internal analysis rather than republication. Consult legal counsel before the system goes live.
Frequently Asked Scenarios: Allowed or Not?
To make this concrete, let us examine some common scenarios with an honest assessment.
"I take competitor product prices for internal analysis"
Small volumes, non-personal data, internal analysis: this is the zone most often considered tolerable, especially for data genuinely displayed publicly. Risk arises if the site's ToS prohibits it, volume overloads servers, or the data is used for systematic imitation. Limit and document.
"I scrape customer contact data from a marketplace to build prospects"
This is clearly problematic. Other people's personal data, taken without consent, for commercial purposes — violates the PDP Law and carries criminal risk. Do not do it, and never buy "prospect databases" scraped and sold on the black market. To find prospects lawfully, use legitimate channels and data obtained with consent, as discussed in our UMKM digital transformation guide — including building healthy customer records from the start.
"I copy news articles for my aggregator site"
To do this legally, you must cooperate with publishers, use official APIs, or use licensed subscription feeds. Direct copying is copyright infringement.
"I take government statistics for a research report"
Official open data exists precisely for this. No problem, provided you cite the source as required.
"I use bots to buy limited-edition goods faster than humans"
One of the most well-known abuses: sneaker bots flood checkout, harming consumers and stores. Many platforms now explicitly prohibit them and reserve the right to cancel orders. It is a risky business and ethically questionable.
Building a Responsible Data Collection System
If your business genuinely needs automated data collection, build it as a responsible system from the start, not as a hidden script. The components of a healthy system:
- Source documentation: every dataset records its origin, collection date, legal basis (official API, open data, license, or limited scraping), and purpose.
- Retention policy: data is discarded when no longer needed, especially personal data. The PDP Law requires the minimization principle: collect only what is necessary, store it only as long as necessary.
- Internal compliance: one person or team is responsible for periodic compliance review, including checking source ToS and regulatory developments.
- Technical limits: rate limiting, polite pauses, robots.txt support, and automatic stop when a site blocks you.
- Shutdown plan: if the site owner asks you to stop, the system can be halted quickly without losing lawfully acquired data.
Also consider how this system interacts with your other infrastructure. If your data collection connects to your website or business applications, make sure the whole chain is built with good security and reliability practices, because a hacked data collection system can become a tool for stealing other people's data.
How Technology Can Help, Not Become the Problem
Technology's role here is often misunderstood. Scraping is not "evil" technology; it is a tool. Its impact is determined by design and purpose. A good technical team can build a polite, documented data collector — or, without oversight, a rogue script that endangers the company.
For this reason, the decision to build a data collection system must not be purely technical. Involve legal and business considerations from the start. This is the same pattern as other technology decisions: when you need an IT consultant is often exactly at the point where the risk is initially invisible — like collecting "seemingly public" data.
One question always worth asking before building: can this data be obtained through official channels, and what would it cost? If the answer is yes, take that path. If not, the next question: can the need be met with other lawful data? Often the answer is yes, and the company saves itself from large legal costs down the road.
A Practical Summary: Golden Rules of Scraping for Business
- Personal data? Do not scrape. Use APIs, consent, or licensed services. The PDP Law closes this gap.
- Copyrighted content? Do not republish. Internal analysis with modest quotation differs from full copying.
- Site prohibits scraping in ToS? Treat it as a serious warning; find another path or seek legal advice.
- Respect robots.txt and server limits. Scraping that overloads servers can be classified as an attack.
- Document everything. Who collected, what, from where, when, and why.
- Never bypass technical protections. Forced login, CAPTCHA, and protected APIs are clear red lines.
- When in doubt, ask — and ask before, not after.
Closing: Data Is Not a Free Good
The phrase "the data is public" is the most expensive misconception in the data world. Public means visible, not necessarily free to harvest in bulk, store, and use for commercial purposes. Indonesian law — the ITE Law, the PDP Law, the Copyright Law, and contract law — increasingly makes clear that data is an asset with owners and rules.
This is not bad news for business. Quite the opposite: legal certainty makes competition healthier. Companies that obtain data through lawful channels no longer have to compete with cheaters; and properly acquired data has defendable legal value, for example when you use it for decisions or reporting.
If your business needs a data collection system — market research, price monitoring, information aggregation — let us build it the right way from the start. The Kartech team in Bandar Lampung has experience designing lawful, documented, and ethical data systems: from mapping official data sources and API integration to analysis dashboards. Reach us through the contact page or explore our services to see how we work.
