Please, support theAnimeGallery.com by visiting our sponsors.
Close
theAnimeGallery.com
 
 
Download our Toolbar with RSS feeds
Home » Forum » News » Checking all images on the site
Checking all images on the site Forums
12 posts and 193 views.
IGC MOD IGC level: 40
56,401 points
Checking all images on the site by IGC on Wed Jul 08th, 2009 - 10:07 am

As you probably know, the site crashed yesterday... the reason: I wrote a script that checks all images to see if the original image, the thumbnail and the other resolutions images are in fact in the site, or were lost in the last move.

Since we've had a lot of problems reported with Anime Cyclone, I thought that was the cause of the problem, and I think it might just be. I have only processed half of the images, and there were a lot of missing resolutions, even though they were listed on the database as available.

Since the script checks to see if the image is in the server, and if not creates it, this was taking huge chunks of processing power and they shut the site down, reasonably so.

I have tweaked the script, so it goes 10 a time, instead of 1000 images a time, to avoid too much load on the server, the only problem is that it will take some time before we finish, since I have to click "Next" every time.

Anyhow, doing this should help with the Anime Cyclone problems as well as with images missing from the site, specially since we are missing more images than I had expected...

Organize your anime! => Anime List Builder v3.0

LuckyKallen LuckyKallen level: 13
6,143 points
Re: Checking all images on the site by LuckyKallen on Wed Jul 08th, 2009 - 07:59 pm

When the site crashed it read: 403 forbidden and I first freak that I was banned or something rofl ^^ but then I figured something might have happened because I don't think I'm that annoying haha

aragorn4000 MOD aragorn4000 level: 23
18,810 points
Re: Checking all images on the site by aragorn4000 on Thu Jul 09th, 2009 - 01:34 am

So that's why theanimegallery was forbidding me to visit the site... How long will the script finish checking the images?

Battle Angel Alita fan 1989 Battle Angel Alita fan 1989 level: 28
27,582 points
Re: Checking all images on the site by Battle Angel Alita fan 1989 on Fri Jul 10th, 2009 - 10:33 am

You guys should be used to it by now heck if it says I can't access the page at least once a month something would feel wrong. But also you guys shouldn't worry because (if you payed attention) over the course of the many times the site has gone down a few times it has said on the page 403 forbidden before. So that should clue you in right there that your not banned.

IGC MOD IGC level: 40
56,401 points
Re: Checking all images on the site by IGC on Fri Jul 10th, 2009 - 10:55 am

It wasn't the script itself, but due to the high CPU usage from running that script, we did cause some trouble on the server and they shut us down for a while :(

Hopefully, everything will work fine now...

If you get the 403 forbidden error, it is the hosting company who has disabled access... If it were me, you would see something different, either the maintenance page, or the banned page: http://www.theanimegallery.com/excessiveUse.html

Organize your anime! => Anime List Builder v3.0

2+2=5 MOD 2+2=5 level: 50
89,179 points
Re: Checking all images on the site by 2+2=5 on Fri Jul 10th, 2009 - 09:54 pm

so, while we are on the subject, IGC doesn't ban ppl. even windowshoppa, which downloaded all the sites images in a day, didnt get banned.

anyway, glad to see the site back up and that it was a trivial reason unlike we have had in the past. go theanimegallery! :)



My Top 10 anime list :D (revised 5-4-09)

IGC MOD IGC level: 40
56,401 points
Re: Checking all images on the site by IGC on Fri Jul 10th, 2009 - 10:27 pm

@2+2=5... your first statement is not completely accurate... ;)

I have a list of almost 30 IPs that I have banned since the site has been up.... However, if they have a user account, and when I email them, they respond and seem apologetic or didn't cause too much stress in the server, then I let it go... If he did already download anything, there is no much damage I can do banning him...

The problem is people or bots that flood the site with dozens of request at the same time, using tons of resources and getting us in trouble for using too much resources. Those get banned in seconds, and if they don't email, they don't get removed.

Organize your anime! => Anime List Builder v3.0

aragorn4000 MOD aragorn4000 level: 23
18,810 points
Re: Checking all images on the site by aragorn4000 on Fri Jul 10th, 2009 - 11:20 pm

The problem is people or bots that flood the site with dozens of request at the same time, using tons of resources and getting us in trouble for using too much resources. Those get banned in seconds, and if they don't email, they don't get removed.

Ohhh... Sounds like a good thing to remember... Anyhow, how many images were lost in the previous move?

IGC MOD IGC level: 40
56,401 points
Re: Checking all images on the site by IGC on Sat Jul 11th, 2009 - 10:38 pm

mmmm, I don't have exact numbers, because I did not have a counter on the script, but there were a bunch... probably two or three hundred.

Organize your anime! => Anime List Builder v3.0

aragorn4000 MOD aragorn4000 level: 23
18,810 points
Re: Checking all images on the site by aragorn4000 on Sun Jul 12th, 2009 - 06:42 am

mmmm, I don't have exact numbers, because I did not have a counter on the script, but there were a bunch... probably two or three hundred.

Whoa... That's alot of images... Can you retrieve them, IGC?

Backend derived from 4homepages.de
Advertise | Privacy | Contact IGC - ©2007
theAnimeGallery.com Forums: Checking all images on the site replies