Media Companion

Discussion and development board for the program Media Companion.
 
HomeHome  FAQFAQ  SearchSearch  RegisterRegister  Log inLog in  

Share | 
 

 Updated IMDB scraper library - 09-October-2010

View previous topic View next topic Go down 
AuthorMessage
billyad2000
Admin
avatar

Posts : 1326
Join date : 2008-09-20

PostSubject: Updated IMDB scraper library - 09-October-2010   Sat Oct 09, 2010 10:49 pm

Just replace the imdb.dll file in your Media Companion folder with the one contained within the download archive.

IMDB made some pretty big changes this time, pretty much had to rewrite every parameter.

It seems that IMDB is being rather clever in that what data you get back will very much depend on where in the world you live. This makes it impossible to return english titles in france for example. As a workaround I have extended the alternative title feature to list most alternative titles. This also makes it impossible for me to test thoroughly, I can only be sure that this works in the UK - although I am hopeful that the html is the same for all locations.

File removed - MC 3.400 has an updated imdb.dll that should fix the issues within this version.

_________________
Do you have a problem with Media Companion, before posting a bug then please check you have the very latest version of Media Companion http://billyad2000.darkbb.com/announcements-f1/


Last edited by billyad2000 on Mon Oct 11, 2010 2:16 am; edited 3 times in total
Back to top Go down
View user profile http://billyad2000.co.uk
jz1276
Junior Member
Junior Member


Posts : 42
Join date : 2009-06-01

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sat Oct 09, 2010 11:45 pm

thanks
Back to top Go down
View user profile
trogggy
New User
New User


Posts : 5
Join date : 2010-10-07

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 12:43 am

The workaround works around in France.
You have to select an alternative title, and the only oddity is that the top alternative title (the one normally needed) seems to have a couple of spaces in front of it - not an issue at all really.
Many thanks. This is a brilliant piece of software!
Back to top Go down
View user profile
trigger.hippie
Media Companion Supporter
Media Companion Supporter


Posts : 3
Join date : 2009-11-13

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 12:55 am

thank you again, billy
Back to top Go down
View user profile
hewwra
New User
New User


Posts : 1
Join date : 2010-10-10

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 2:08 am

I tried the new dll and it works for all but one of my recently added movies.
When I try and scrape 'Predators' I get this in the error logfile:

Starting Rescrape

Deleting existing poster and backdrops

Clearing current movie details

Scraping Movie Body with settings:
tt1424381
http://www.imdb.com/


System.Xml.XmlException: Reference to undeclared entity 'oacute'. Line 7, position 16.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.Throw(String res, String arg, Int32 lineNo, Int32 linePos)
at System.Xml.XmlTextReaderImpl.HandleGeneralEntityReference(String name, Boolean isInAttributeValue, Boolean pushFakeEntityIfNullResolver, Int32 entityStartLinePos)
at System.Xml.XmlTextReaderImpl.ResolveEntity()
at System.Xml.XmlLoader.LoadEntityReferenceNode(Boolean direct)
at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace)
at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc)
at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace)
at System.Xml.XmlDocument.Load(XmlReader reader)
at System.Xml.XmlDocument.LoadXml(String xml)
at Media_Companion.Form1.Button21_Click(Object sender, EventArgs e)

End of log
Back to top Go down
View user profile
billyad2000
Admin
avatar

Posts : 1326
Join date : 2008-09-20

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 2:38 am

hewwra wrote:
I tried the new dll and it works for all but one of my recently added movies.
When I try and scrape 'Predators' I get this in the error logfile:

I've identified the issue and i'll upload a fix for it tomorrow.

_________________
Do you have a problem with Media Companion, before posting a bug then please check you have the very latest version of Media Companion http://billyad2000.darkbb.com/announcements-f1/
Back to top Go down
View user profile http://billyad2000.co.uk
shadeblack
New User
New User
avatar

Posts : 9
Join date : 2010-03-12
Age : 28
Location : england - essex - basildon

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 12:19 pm

took me some time to scrape a movie, but it's working.

thanks, billy!
Back to top Go down
View user profile
billyad2000
Admin
avatar

Posts : 1326
Join date : 2008-09-20

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 1:05 pm

The scraper will be a little bit slower - a couple of extra webpages need to be loaded per scrape.

The certification data has been moved away from the main page
and
Since users can no longer select their chosen language another page needs to be loaded containing alternative titles.

The biggest issue though is not caused from the above, or at least not directly. Since the changes to IMDB, the website speed has dropped dramatically. This can be seen when viewing IMDB thumbnails, it has moved from the fastest source to one of the slowest.

_________________
Do you have a problem with Media Companion, before posting a bug then please check you have the very latest version of Media Companion http://billyad2000.darkbb.com/announcements-f1/
Back to top Go down
View user profile http://billyad2000.co.uk
Nukhem
New User
New User


Posts : 4
Join date : 2009-04-13

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 3:08 pm

It seems i'm unable to get 'Machete (2010)' to scrape.

Keep up the good work
Back to top Go down
View user profile
genial
New User
New User


Posts : 3
Join date : 2010-10-10

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 11:03 pm

Everything but movies plots scrape fine now.
Plots turn up empty.

Keep up the awesome!
Back to top Go down
View user profile
billyad2000
Admin
avatar

Posts : 1326
Join date : 2008-09-20

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 11:08 pm

genial wrote:
Everything but movies plots scrape fine now.
Plots turn up empty.

Keep up the awesome!

What location are you scraping from?

_________________
Do you have a problem with Media Companion, before posting a bug then please check you have the very latest version of Media Companion http://billyad2000.darkbb.com/announcements-f1/
Back to top Go down
View user profile http://billyad2000.co.uk
genial
New User
New User


Posts : 3
Join date : 2010-10-10

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 10, 2010 11:12 pm

billyad2000 wrote:
genial wrote:
Everything but movies plots scrape fine now.
Plots turn up empty.

Keep up the awesome!

What location are you scraping from?

Scraping from Norway, through the www.imdb.com mirror.
Back to top Go down
View user profile
genial
New User
New User


Posts : 3
Join date : 2010-10-10

PostSubject: Re: Updated IMDB scraper library - 09-October-2010   Sun Oct 17, 2010 6:06 pm

genial wrote:
Everything but movies plots scrape fine now.
Plots turn up empty.

Keep up the awesome!
Turns out that the movies I tried to scrape:

- The Hole 2009
- Get Him To The Greek 2010
- Grown Ups 2010

don't have plot summaries on IMDB...
Woops :p
Back to top Go down
View user profile
angeoand
Junior Member
Junior Member


Posts : 25
Join date : 2011-03-11

PostSubject: Dress for pregnant Bridesmaid?   Sun May 08, 2011 10:59 am

alien alien I'm so like your style.I agree to your opinion.Hoping
more your better article!Pearl Jewelry
Back to top Go down
View user profile
wedd123
Junior Member
Junior Member


Posts : 25
Join date : 2011-05-12

PostSubject: crystal earrings   Sat May 14, 2011 9:31 am

The workaround works around in France.
You have to select an alternative title, and the only oddity is that the top alternative title (the one normally needed) seems to have a couple of spaces in front of it - not an issue at all really.
Many thanks. This is a brilliant piece of software!
crystal earrings

Crystal Brooch
Back to top Go down
View user profile
Sponsored content




PostSubject: Re: Updated IMDB scraper library - 09-October-2010   

Back to top Go down
 
Updated IMDB scraper library - 09-October-2010
View previous topic View next topic Back to top 
Page 1 of 1
 Similar topics
-
» 2010 Bulwer-Lytton Fiction Contest Winners
» Headache 2010-2011
» 2010: My year for The Migraine Problem: Now Gabapentin (Neurontin)
» XP Internet Security 2010
» Softswitch MediaCore-winner of TMC 2010 Award

Permissions in this forum:You cannot reply to topics in this forum
Media Companion :: Media Companion :: Announcements-
Jump to: