Recording live scores (Web scraping)

  • Thread starter Thread starter AnonConfession
  • Start date Start date
Status
Not open for further replies.
A

AnonConfession

Guest
I have written a computer program to record live scores for my favourite sport from the Internet. I post the scores on the Internet after the matches have finished, and I do not charge any money for this service. I am now worrying about the morality of recording and posting scores.

Here are all the possible moral issues that I can think of:
    • Time-wasting. I often spend 2–4 hours a day processing these scores, which means that I am committing various sins of omission.
    • Copyright. The terms and conditions of the Websites from which I record live scores tend to say that I must not “copy”, “record”, “download”, “store”, “reproduce”, “publish” or “post” any part of their content. I believe that such prohibitions are intended to protect articles, photos, videos, and any other original expressions on these sites, whereas scores are just simple historical facts (though one site does specifically mention “tournament scores”). However, I do worry about whether I could be violating copyright by posting the scores elsewhere, or even just by recording them.
    • Bandwidth. My program reloads the scores from each site once every 10 seconds (to minimise missing scores), whereas a normal Web browser reloads them once every 30 seconds. Each download is less than 100 KB. Given that many other (thousands?) of fans of my sport are viewing them normally, I don’t think recording live scores comes anywhere close to throttling the servers, or even “overuse or misuse of the Services”. But I am consuming more than my fair share of their bandwidth.
    • Sunday rest. I record, process and post scores even on Sundays and other Holy Days of Obligation. I worry that I might be violating the obligation of rest by doing so – if not my own rest, then I might be committing scandal by supporting a sport that forces many people to work on Sundays.
    • Gambling. I might possibly be encouraging gambling (which is legal in my country but against my principles) by posting the scores, if gamblers could potentially discover useful patterns in them. They do occasionally ask me for missing scores to resolve disputes with their bookmakers.
    I haven’t asked the sites from which I record live scores for permission, because I wouldn’t expect a positive response, and am paranoid that I could be told to “cease and desist”, or even be prosecuted for “copyright infringement” or “trespass to chattels”, as other Web scrapers have been (not specifically for recording live scores). One site started encrypting its scores, which implies that the owner didn’t like them being recorded.

    I accept the time-wasting as a venial sin, and accept that I’ll have to cut down on processing scores in order to manage some imminent new responsibilities in my life. My conscience tells me that I’m probably just being scrupulous (as usual) about Issues 2–5, but if I’m going to confess recording live scores sacramentally, then I need to know exactly what I should be confessing.

    I am considering three options:
      • Confess the time-wasting, but withhold Issues 2–5; continue to record live scores and post them on the Internet, but try to limit my manual post-editing to two hours a day.
      • Confess the time-wasting and maybe some of Issues 2, 4, 5; continue to record live scores for my own personal records, but stop processing them and posting them on the Internet.
      • Confess everything, and stop recording live scores.
      So, what I’m asking is:
      • Which of Issues 2–5, if any, should I confess in the Sacrament of Reconciliation?
      • Do any of the five issues constitute grave matter for mortal sin?
      • Can you think of any other sins that I might be committing by recording live scores and posting them (free of charge) after the matches have finished?
 
No sin. Sports scores are news. You should talk to an experienced priest about your scrupulosity.
 
Scrupulosity is not the issue here. Just contact - anonymously - any sports team or owner and ask if what you are doing is allowed. Simple.

No need to worry since finding out will answer your question(s).

Peace,
Ed
 
I don’t know the law where you live (in England), but in the US, what you are doing would probably be legal under the principle of fair use. I assume you are not archiving the recordings, but merely reviewing, summarizing the scores, and discarding the recordings. This is not much different from someone recording a program in order to view it at a more convenient time. I am pretty sure the scores themselves are not copyrighted or legally protected in any way.

I cannot advise you on the sinfulness of this activity, but I think you would do very well to explore other hobbies and interests. You may have an obsessive/compulsive personality trait, which is not necessarily a bad thing; if you focused this amazing attention and energy in other areas, you might find it much more fulfilling. What else would you like to do with your time and energy?
 
I am archiving the recorded scores.

More precisely, my program:


  1. *]downloads an HTML document every time it reloads the scores;
    *]extracts the scores from the HTML document;
    *]saves the extracted scores in a text file (the same file for all the scores in that session).

    I discard the original HTML documents (except for a few samples that I use to develop my program), but keep the text files forever – and post them on the Internet.
 
I can’t see how what you’re doing is, in any way, different to what Google or Bing do when their ‘spiders’ look at sites for search engine archival.

The only thing I would do is ensure that I respected whatever the terms were of the ‘robots.txt’ file on each site if they have one.

I can’t see how a ‘fact’ can be copyrighted, and presumably you’re not stealing their bandwidth by making use of it whenever someone goes to your site (i.e by using their file assets ‘real time’ in the web pages that you serve), so I can’t see a problem with anything you’re doing.
 
Incidentally, your poll is missing an option for those who don’t think there’s a sin!
 
DexUK said:
The only thing I would do is ensure that I respected whatever the terms were of the ‘robots.txt’ file on each site if they have one.
Click to expand...
The robots.txt file of my favourite site from which I record live scores looks like this:
Code: 
User-Agent: *
Disallow: /livescore.php
This tells all robots to stay out of the live-scores page.

So, is it a sin to record live scores from this site? If so, how do I confess it?
 
I’m still not sure that ignoring a ‘robots.txt’ file counts as a sin per se. Recounting a fact is simply recounting a fact, wherever you learned the fact from. Otherwise you’d be sinning just by reading the page and retaining the memory of it in your head or jotting it down on a scrap of paper or telling someone else over the phone what you’re reading off their web page! They are putting this information into the public domain, after all, aren’t they? And I presume that the information is made available freely - i.e. you’re not disseminating facts that others could ONLY get by paying a fee?

You may find that you’re in breach of some contract law, I suppose, based on what the purveyor of the facts licenses you to do with their material and the permissions they give you to use their resources (the website) and the purpose to which you put the information, but I struggle to see that going so far as a ‘theft’. If it was, you might just scrape into something venial I suppose.

Theft would be making use of their resources - i.e. a jpg or gif file of theirs that you embed in your html code - in such a way that it requires use of their bandwidth every time one of your website visitors loads one of your pages.
 
In terms of stealing bandwidth, what’s the moral difference between recording live scores and hotlinking images?
  • Both use others’ bandwidth without permission. A quick calculation suggests that recording live scores could be using bandwidth on a scale of tens of gigabytes per month, and I do feel quite guilty about that. However, I have no idea whether this is actually a significant percentage of their monthly data transfer.
  • I’m the only one using their bandwidth to record live scores. I have not shared my program with others for this reason.
  • Many people viewing a page containing hotlinked images could potentially use much more bandwidth than recording live scores, depending on the number and sizes of the images, and how many times they are viewed.
 
Status
Not open for further replies.
Back
Top