Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[wanttoknow]

We have an online service website. In our term of use, we clearly state that
user can not use any manual or automate process to crawl/download/collect our
data from our website.

However, our web log shows that a company, say company A, recently intensively
crawls our data. The IP address of company A shows that it is a big famous
company (market capitcal is $30 Billion). That means someone from company A
illegally crawl our data.

How can we enforce our term of use? Can we file a lawsuit against
company A?

Thanks a lot for your help!

[artchain]

I've read of a few court cases that have held that reasonable "Terms of Use" are enforceable, including Terms of Use designed to restrict competitive use of a site.

However, before pursuing legal action, I'd try these "first steps" :

(1)  Configure your robots.txt file to block the crawler.  Responsible companies follow robots.txt restrictions.

(2)  If the crawler continues to access your site, send a certified letter to the company stating that they are in violation of your Terms of Use, and requesting that they immediately stop crawling your site.

[wanttoknow]

artchain,

Thanks for the info. We do have robots.txt file.

Actually, the web URL being illegally crawled cannot be found by the normal web crawlers because they are written in javascript. Only hackers can figure out those URL manually and start to crawl our data. Can this fact be used as an evidence?

Thanks!

[Isaac]

Only hackers can figure out those URL manually and start to crawl our data. Can this fact be used as an evidence?

Does hacking mean using "view source" from internet explorer or something more sinister?  Java script is not exactly hieroglyphics.

Terms of service might not be binding in situations where a user has neither seen nor agreed to them.   There still might be some law you can use that covers unauthorized access to and use of computer resources, particularly if the use of your site is egregious.

If you don't allow people to "manually crawl your site", then how do people use it?

[artchain]

I don't think anyone can assess the strength of your case based on a few posted facts.  While some reasonable "Terms of Use" have been deemed enforceable, courts have overturned others.  As Isaac noted, one factor that is considered is how prominently the Terms of Use are presented, and whether the unauthorized user has seen them.

As I said, I would first try to block the crawler with the robots.txt file, and as a second step, send a certified letter requesting that the company stop crawling your site.

If those measures fail, then you may want to consider a lawsuit.  You'll need to find an attorney who is knowledgeable about web "Terms of Use" and also the Computer Fraud and Abuse Act.  You should be able to get a referral for an inexpensive consultation through your local bar association.

Bear in mind that, if litigation is required, it's likely to be expensive.  You need to think about how you are being "damaged" when your site is crawled, and whether the issue is worth pursuing.  Again, a consultation with an attorney will help clarify the options and likelihood of success.

[wanttoknow]

Issac and Artchain,

Thank you for you reply! We appreciate your help and will consult with some attorney first on this case.