| | Updated IMDB scraper library - 09-October-2010 | |
|
| Author | Message |
|---|
billyad2000 Admin

Posts: 1326 Join date: 2008-09-20
 | Subject: Updated IMDB scraper library - 09-October-2010 Sat Oct 09, 2010 8:49 pm | |
| Just replace the imdb.dll file in your Media Companion folder with the one contained within the download archive.
IMDB made some pretty big changes this time, pretty much had to rewrite every parameter.
It seems that IMDB is being rather clever in that what data you get back will very much depend on where in the world you live. This makes it impossible to return english titles in france for example. As a workaround I have extended the alternative title feature to list most alternative titles. This also makes it impossible for me to test thoroughly, I can only be sure that this works in the UK - although I am hopeful that the html is the same for all locations.
File removed - MC 3.400 has an updated imdb.dll that should fix the issues within this version.
Last edited by billyad2000 on Mon Oct 11, 2010 12:16 am; edited 3 times in total |
|
 | |
jz1276 Junior Member

Posts: 42 Join date: 2009-06-01
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sat Oct 09, 2010 9:45 pm | |
| |
|
 | |
trogggy New User

Posts: 5 Join date: 2010-10-07
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sat Oct 09, 2010 10:43 pm | |
| The workaround works around in France. You have to select an alternative title, and the only oddity is that the top alternative title (the one normally needed) seems to have a couple of spaces in front of it - not an issue at all really. Many thanks. This is a brilliant piece of software! |
|
 | |
trigger.hippie Media Companion Supporter

Posts: 3 Join date: 2009-11-13
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sat Oct 09, 2010 10:55 pm | |
| |
|
 | |
hewwra New User

Posts: 1 Join date: 2010-10-10
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sun Oct 10, 2010 12:08 am | |
| I tried the new dll and it works for all but one of my recently added movies. When I try and scrape 'Predators' I get this in the error logfile:
Starting Rescrape
Deleting existing poster and backdrops
Clearing current movie details
Scraping Movie Body with settings: tt1424381 http://www.imdb.com/
System.Xml.XmlException: Reference to undeclared entity 'oacute'. Line 7, position 16. at System.Xml.XmlTextReaderImpl.Throw(Exception e) at System.Xml.XmlTextReaderImpl.Throw(String res, String arg, Int32 lineNo, Int32 linePos) at System.Xml.XmlTextReaderImpl.HandleGeneralEntityReference(String name, Boolean isInAttributeValue, Boolean pushFakeEntityIfNullResolver, Int32 entityStartLinePos) at System.Xml.XmlTextReaderImpl.ResolveEntity() at System.Xml.XmlLoader.LoadEntityReferenceNode(Boolean direct) at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace) at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc) at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace) at System.Xml.XmlDocument.Load(XmlReader reader) at System.Xml.XmlDocument.LoadXml(String xml) at Media_Companion.Form1.Button21_Click(Object sender, EventArgs e)
End of log |
|
 | |
billyad2000 Admin

Posts: 1326 Join date: 2008-09-20
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sun Oct 10, 2010 12:38 am | |
| | hewwra wrote: | I tried the new dll and it works for all but one of my recently added movies. When I try and scrape 'Predators' I get this in the error logfile: |
I've identified the issue and i'll upload a fix for it tomorrow. |
|
 | |
shadeblack New User


Posts: 9 Join date: 2010-03-12 Age: 22 Location: england - essex - basildon
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sun Oct 10, 2010 10:19 am | |
| took me some time to scrape a movie, but it's working.
thanks, billy! |
|
 | |
billyad2000 Admin

Posts: 1326 Join date: 2008-09-20
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sun Oct 10, 2010 11:05 am | |
| The scraper will be a little bit slower - a couple of extra webpages need to be loaded per scrape.
The certification data has been moved away from the main page and Since users can no longer select their chosen language another page needs to be loaded containing alternative titles.
The biggest issue though is not caused from the above, or at least not directly. Since the changes to IMDB, the website speed has dropped dramatically. This can be seen when viewing IMDB thumbnails, it has moved from the fastest source to one of the slowest. |
|
 | |
Nukhem New User

Posts: 4 Join date: 2009-04-13
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sun Oct 10, 2010 1:08 pm | |
| It seems i'm unable to get 'Machete (2010)' to scrape.
Keep up the good work |
|
 | |
genial New User

Posts: 3 Join date: 2010-10-10
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sun Oct 10, 2010 9:03 pm | |
| Everything but movies plots scrape fine now. Plots turn up empty.
Keep up the awesome! |
|
 | |
billyad2000 Admin

Posts: 1326 Join date: 2008-09-20
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sun Oct 10, 2010 9:08 pm | |
| | genial wrote: | Everything but movies plots scrape fine now. Plots turn up empty.
Keep up the awesome! |
What location are you scraping from? |
|
 | |
genial New User

Posts: 3 Join date: 2010-10-10
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sun Oct 10, 2010 9:12 pm | |
| | billyad2000 wrote: | | genial wrote: | Everything but movies plots scrape fine now. Plots turn up empty.
Keep up the awesome! |
What location are you scraping from? |
Scraping from Norway, through the www.imdb.com mirror. |
|
 | |
genial New User

Posts: 3 Join date: 2010-10-10
 | Subject: Re: Updated IMDB scraper library - 09-October-2010 Sun Oct 17, 2010 4:06 pm | |
| | genial wrote: | Everything but movies plots scrape fine now. Plots turn up empty.
Keep up the awesome! |
Turns out that the movies I tried to scrape:
- The Hole 2009 - Get Him To The Greek 2010 - Grown Ups 2010
don't have plot summaries on IMDB... Woops :p |
|
 | |
angeoand Junior Member

Posts: 25 Join date: 2011-03-11
 | Subject: Dress for pregnant Bridesmaid? Sun May 08, 2011 8:59 am | |
|  I'm so like your style.I agree to your opinion.Hoping more your better article!Pearl Jewelry |
|
 | |
wedd123 Junior Member

Posts: 25 Join date: 2011-05-12
 | Subject: crystal earrings Sat May 14, 2011 7:31 am | |
| The workaround works around in France. You have to select an alternative title, and the only oddity is that the top alternative title (the one normally needed) seems to have a couple of spaces in front of it - not an issue at all really. Many thanks. This is a brilliant piece of software! crystal earrings
Crystal Brooch |
|
 | |
| | Updated IMDB scraper library - 09-October-2010 | |
|