menu

Scrape Away... 85 views

This is great news. Been wanting to automate game schedules and lineups for MLB and NHL for some time but have always been put off by not wanting to check code in that is'nt "legal".


This ruling should make it easier to ask questions about scraping as well as you can see at the following link that the first response can sometimes be "you should'nt"

https://codereview.stackexchange.com/a/193421


However, it's a double edge sword. I wonder how PickMonitor feels about game lines being scraped from this site without collecting money for that service?

3

Hmm.. Not sure there is any value to scraping lines? I guess i dont know if you could actually scrape the feed either. Likely you would just get old and out of date lines? Maybe I just dont know but it seems pointless. Isnt there a widget for that anyway? (legit asking because i dont know.) I would say thought that Scraping questions may not get you anywhere and keep in mind that a simple code change could invalidate all of your work.


2

I think feeding the line into an algorithm is pretty important. Scraping lines can help determine if a wager is worth placing. Without that information, most algorithms would always be having you place bets on -300 games (not arguing against that). While it might be slightly out of date and have moved a bit, unless a player gets injured or some outside force acts on the line it wont move too much from whatever value you gathered to make your decision. And scraping lines from the right source is even more important, especially if it is where you will ultimately be placing a wager. With live betting getting more attention I could see a sort of high frequency trading situation happening where you want to jump on a wager automatically when a favorite gets down by a few points/runs early in the game and the line becomes more favorable.

3

Scraping has always been a grey area, and if companies don't want you to scrape, they can block your IP, limits your queries, file a lawsuit etc. The other issue is by scraping, if the website changes, it can easily break your ability to scrape data.

Scraping has many benefits for websites which don't change and there is no API to gather the data. How this relates to sports is any reasonable API call for updated scores is super expensive.

The good news is, it's not illegal to do so going forward and it has legitimate uses. Of course, your IP could always end up getting blocked or banned in some other fashion.

3

Totally agree with your points, but in cases where sites want and need to display information to its users, for example if a wagering site needs to show you what the line is for a given wager, then you can always figure out a way to scrape it no matter what they do. You can always use multiple machines in various geo's to find the data, and you can be smarter about your scraping to make it more about what is on the screen versus just looking for tags to pull that data out.


Even for more general users where they just want to feed a site into excel using the built in "import from web page" it seems like maybe now that its not illegal, Microsoft and others can put more efforts into making it easier for users to import data that is not just in tables now. There is a huge push for data science and machine learning so I feel like more and more people will be supporting this type of scraping now. I have been able to get pitchers ERA's into google sheets directly from the web without too much effort and I would like to see more of this.

2

Please Login or Register to reply.

Ready to start winning?

Yes, Sign me up now No, show me the handicappers first