Media Companion

Discussion and development board for the program Media Companion.
 
HomeHome  FAQFAQ  SearchSearch  RegisterRegister  Log inLog in  

Share | 
 

 IMDB Scraper fix - 19th November 2009

View previous topic View next topic Go down 
AuthorMessage
billyad2000
Admin
avatar

Posts : 1326
Join date : 2008-09-20

PostSubject: IMDB Scraper fix - 19th November 2009   Thu Nov 19, 2009 1:16 pm

Fixed IMDB scraper library,
This will also fix the issue with spec characters.

Download imdb.rar
I've tested a few movies with this and it seems fine, but there is always the possibility of a discrepancy in a few movies, if anyone has any issues then let me know, be sure to include the imdb url to the problem movie.

Also available is tmdb_posters.rar, this has been updated to use TMdb API 2.1, it seems to be significantly faster than 2.0
Download tmdb_posters.rar

Download the dll's and extract the .dll files to your MC directory, when asked just replace the current files.

_________________
Do you have a problem with Media Companion, before posting a bug then please check you have the very latest version of Media Companion http://billyad2000.darkbb.com/announcements-f1/


Last edited by billyad2000 on Mon Nov 23, 2009 12:23 am; edited 1 time in total
Back to top Go down
View user profile http://billyad2000.co.uk
StormyKnight
VIP
VIP


Posts : 556
Join date : 2008-10-08
Location : Australia

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Thu Nov 19, 2009 1:34 pm

Works good billy, seems to be taking about 8 seconds for a complete movie scrape!

PS I appreciate the quick fix. Laughing
Back to top Go down
View user profile
Billy Bunter
Media Companion Supporter
Media Companion Supporter


Posts : 15
Join date : 2008-09-25
Age : 48
Location : Overbury

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Thu Nov 19, 2009 6:29 pm

Fantastic work Billy, I'm getting good results with the two updated .dll's cheers

BB
Back to top Go down
View user profile
Michael Prassel
New User
New User


Posts : 2
Join date : 2009-11-20

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Fri Nov 20, 2009 3:22 am

Thank you so much for all of your work. It's working great.
Back to top Go down
View user profile
goonie
New User
New User


Posts : 5
Join date : 2009-11-20

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Fri Nov 20, 2009 6:24 am

fixed quite a few problems, but I'm noticing that some taglines and almost all runtimes have scraper error now? Using 3.154 with these two updates.
Back to top Go down
View user profile
paybac
Media Companion Supporter
Media Companion Supporter
avatar

Posts : 155
Join date : 2009-08-31
Location : New Zealand

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Fri Nov 20, 2009 7:06 am

I'm not having any problems just scraped 4 movies. Are you sure u upgraded to 3.154 then did the 2 .dll updates? or did you do the two updates then upgrade to 3.154?
Back to top Go down
View user profile
esra
Junior Member
Junior Member


Posts : 45
Join date : 2009-05-08

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Fri Nov 20, 2009 7:27 am

Yep, just tested with 3 movies and all are fine.
Back to top Go down
View user profile
goonie
New User
New User


Posts : 5
Join date : 2009-11-20

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Fri Nov 20, 2009 7:46 am

i was able to delete all of the nfo files manually and then rerun the find movies to create new nfos. seems to work ok now. only setting I changed in the preference was to get the paths for all the thumbnails from all four sites... but didnt test the reverse to see if that was the problem or that my nfo's were from an older version that was incompatible.
Back to top Go down
View user profile
billyad2000
Admin
avatar

Posts : 1326
Join date : 2008-09-20

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Fri Nov 20, 2009 1:13 pm

IMDB is doing something wierd at the moment, perhaps it is only effecting certain mirrors.

Either way, i'll give them a day or two to complete whatever it is they are doing

_________________
Do you have a problem with Media Companion, before posting a bug then please check you have the very latest version of Media Companion http://billyad2000.darkbb.com/announcements-f1/
Back to top Go down
View user profile http://billyad2000.co.uk
lmaolmao
New User
New User


Posts : 6
Join date : 2009-05-31

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Sat Nov 21, 2009 3:17 pm

Still having a bit of trouble with files which have information after the (year)

Here are some examples:

Rear Window (1954) - 123.mkv - WORKS
Rear Window (1954) - 123456.mkv - WORKS
Rear Window (1954) - 12345678.mkv - WORKS
Rear Window (1954) - 123456789.mkv - FAILS

Rear Window (1954) - abcdefgh.mkv - WORKS
Rear Window (1954) - abcdefghi.mkv - WORKS
Rear Window (1954) - abcdefghij.mkv - WORKS
Rear Window (1954) - abcdefghijk.mkv - FAILS

Rear Window (1954)abcdefghij.mkv - WORKS
Rear Window (1954)abcdefghijk.mkv - FAILS

Rear Window (1954) 01234567.mkv - FAILS
Rear Window (1954) 12345678.mkv - WORKS
Rear Window (1954) 123456789.mkv - FAILS

I still havent worked out the pattern. although it seems like it can only handle 8 numbers or ten letters, after the (year) i haven't tried with combinations of letters and numbers.

When it fails it says:

Code:

Starting Folder Scan

foundG:\test
Checking for subfolders
1 New movies found in directory:- G:\test



1 Movies found in all folders
Obtaining Title for each movie found, from path and filename


Filename is: G:\test\Rear Window (1954) - test 720p.mkv
Title according to settings is: "Rear Window"


Starting Main Scraper Process

Scraping Title:- Rear Window (1954) - test 720p
Output filename:- G:\test\Rear Window (1954) - test 720p.nfo
Poster Path:- G:\test\Rear Window (1954) - test 720p.tbn
Fanart Path:- G:\test\Rear Window (1954) - test 720p-fanart.jpg
NFO does not exist
Checking filename for IMDB ID
No IMDB ID found
Checking for Movie year in filename
Year found for movie:- 1954
Cleaned Title for search :- Rear Window (1954) - test 720p
Unable to scrape body with refs "Rear Window (1954) - test 720p", "", "", "http://www.imdb.com/"
To add the movie manually, go to the movie edit page and select "Change Movie" to manually select the correct movie
Movie added to list
Back to top Go down
View user profile
billyad2000
Admin
avatar

Posts : 1326
Join date : 2008-09-20

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Sun Nov 22, 2009 1:07 pm

lmaolmao wrote:
Still having a bit of trouble with files which have information after the (year)

Here are some examples:

Rear Window (1954) - 123.mkv - WORKS
Rear Window (1954) - 123456.mkv - WORKS
Rear Window (1954) - 12345678.mkv - WORKS
Rear Window (1954) - 123456789.mkv - FAILS

Rear Window (1954) - abcdefgh.mkv - WORKS
Rear Window (1954) - abcdefghi.mkv - WORKS
Rear Window (1954) - abcdefghij.mkv - WORKS
Rear Window (1954) - abcdefghijk.mkv - FAILS

Rear Window (1954)abcdefghij.mkv - WORKS
Rear Window (1954)abcdefghijk.mkv - FAILS

Rear Window (1954) 01234567.mkv - FAILS
Rear Window (1954) 12345678.mkv - WORKS
Rear Window (1954) 123456789.mkv - FAILS

I still havent worked out the pattern. although it seems like it can only handle 8 numbers or ten letters, after the (year) i haven't tried with combinations of letters and numbers.

It appears that the year was not being passed to the scraper, this is fixed for the next build.

I have no idea about the problem regarding the trailing chrs after the year. I checked the code and it is working properly. I am wondering if it is related to the google vs IMDB search. The first 50 searches in a session are performed via google then the rest are via IMDB. google is much more accurate, but used to often it can view MC as a bot and blocks a users IP, to prevent this I limited MC to 50 google searches, this is far below the point when I experienced problems.

Either way, in my tests both imdb and google returned the correct movie for the above filenames, the build will be uploaded later today.

_________________
Do you have a problem with Media Companion, before posting a bug then please check you have the very latest version of Media Companion http://billyad2000.darkbb.com/announcements-f1/
Back to top Go down
View user profile http://billyad2000.co.uk
jz1276
Junior Member
Junior Member


Posts : 42
Join date : 2009-06-01

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Tue Dec 01, 2009 1:22 pm

the movie "Phone Booth" fails to scrape. When I tried to edit the movie, the IMDB page will not laod.
Back to top Go down
View user profile
jz1276
Junior Member
Junior Member


Posts : 42
Join date : 2009-06-01

PostSubject: Re: IMDB Scraper fix - 19th November 2009   Wed Dec 02, 2009 9:37 am

jz1276 wrote:
the movie "Phone Booth" fails to scrape. When I tried to edit the movie, the IMDB page will not laod.

nevermind, my internet settings were off. All's working good.
Back to top Go down
View user profile
Sponsored content




PostSubject: Re: IMDB Scraper fix - 19th November 2009   

Back to top Go down
 
IMDB Scraper fix - 19th November 2009
View previous topic View next topic Back to top 
Page 1 of 1
 Similar topics
-
» Brigg's Galaxy Tablet Sweepstakes
» 2012 Wineries Unlimited Sweepstakes
» nej hmoo puas phem li kuv
» My friend commit suicide last night
» Poj Ntxoog Pw Tav Kev

Permissions in this forum:You cannot reply to topics in this forum
Media Companion :: Media Companion :: Announcements-
Jump to: