FYI

I really noticed the slow down when the new Java Web Start thinggy was added for adding pictures

 

ClubFTE members access the site with a completely different server if memory serves. You want premium access? Its going to cost you $15/yr. No easy way for me to put it.

 

Well if you're right, then my suspicions will subside. I was wondering why my luck was improving during heavy traffic hours.

Ryan

 

I wasn't scolding anyone.

I try and give straight from the hip info !

I thought I gave a complete answer about the system and what & how to do things to overcome some situations.

There are Many Benefits to ClubFTE that NON-ClubFTE users don't get.

This includes special server use which would be eliminate the situation you mention a lot, by buying a ClubFTE membership.


Ken works on the site all the time to improve things.

Thing of problems on FTE like a road crew trying to build an addition to a busy road.

They may have to shut down a lane during off-peak times to allow construction of another part of the road

 

I get the server busy message, but only during peak internet times.

 

The problem is indeed the Java photo uploads. It used to be that only a few photos at a time were uploaded at once.

Now, many photos can be loaded at once. Once they are uploaded the FTE server kicks in and resizes them as needed. I've been watching the server all night and each time the load spikes its when a large set of uploads occurs. I'm looking at possible programming solutions to this and have found one, but it will take a couple of days to code.

 

Ken, out of curiousity, what kind of servers are we running? Dual core platforms, how many hard disks? What kind of specs? I'm kinda curious.

Ryan

 

How many people are uploading? how many pics? Maybe make it a weekend or after hours thing only?

 

Primary database server:
Dual 64-bit Opterons, 6 gigabytes RAM, RAID5 SCSI array running 15K RPM hard drives

Secondary database server:
Dual 32-bit Athlons, 2.5 (maybe three, not sure right off the top of my head) gigabytes RAM, dual 10K RPM Western Digital Raptor drives on SATA RAID controller.

Primary web server:
Dual 32-bit Athlons, 4 gigabytes RAM, 2 RAID 1 arrays running 10K RPM SCSI hard drives.

The above three are interconnected with 1000 Mbit Ethernet network cards. All cards are 66Mhz 64-bit PCI cards for max through-put.

Additionally:

3 gigahertz Intel P4 with 1 gigabytes of RAM runs the ads. No special hardware here, this one doesn't see much load.

450 Mhz Intel PIII with 512 meg RAM runs the images server. This doesn't see much load at all, image serving is very low demand stuff. This is actually an 8 year old Dell box that has been handed down from one task to another. Very reliable, hasn't been rebooted since it came online months ago except when we updated the kernel.

New server will be coming online soon, to handle a variety of tasks. Specs on that will be released when its live.

The images and ads servers are hosted in house, on 2 T1s.

The FTE web/database servers are in a hosting facility down town... they use about as much bandwidth as an additional 2-3 T1s, depending on load.

 

Also, almost forgot... between all these servers we're pushing out well over a terabyte per month of bandwidth (and that's a very conservative estimate). Our daily log files are probably larger than the actual monthly bandwidth of most forums.

 

May I make another suggestion on using the quote feature since I saw it mentioned several times on here, and that is if you quote a person you can edit the quote by deleting all of the irrevelant parts of the original post and only leaving the part you are responding to. I would assume that would also help cut down bandwidth use to some extent, and I personally try and do that whenever I use the quote feature.

 

good to see you have Opterons! Are you using socket 939 or socket 940? That buffered ram costs a lot of money, and I don't think error checking would be terribly necessary for a web-server, but what do I know?

Would a RAID 0 configuration help the cause? The striping helps decrease load times for games with hugely graphical intensive maps, so maybe the same will occur here. I'm not too familiar with RAID 5 and didn't quite get it when I tried reading about it. If you run a RAID 1 in addition to a RAID 0 setup, you can have a double-speed backed-up copy. That'd be pretty dange nifty.

As a geek, Great Job Ken!

Ryan

 

Not sure of the socket type... I think its 940 (that's what requires registered ram, correct?).

Both database servers and the main web server run nothing but registered ECC ram. Its a definite asset knowing that the servers will run even if a memory module starts to fail (and the operating system will log that). Its a one hour drive to the server facility when traffic is good, and I'd rather wait a few days to replace a memory stick at my leasure than deal with a dead server that needs immediate attention. Yeah, it costs twice as much for the memory (and I buy only Corsair brand which is even more expensive), but its worth it when you consider the cost of my time when a server is down.

I won't touch RAID0, it doesn't have any redundancy (it actually increases the odds by 2 that you'll have a failure). RAID1 with a good enterprise level controller gives pretty much the same benefits as RAID0 on reads, but not on writes). I use only LSI Logic 64-bit PCI RAID controllers. The SCSI RAID controller is an enterprise level card, with lots of on board RAM and battery backup built onto the card. RAID5 with a good controler is screaming fast (basically reading from 4 drives at once!) for databases and offers excellant redundancy. You can lose an entire drive and it keeps running....

Most of the load on the web server is CPU, not drive I/O. The graphics resizing really hits it hard. I'm using what's called Imagemagik for image manipulation. Been doing some reading and apparently the GD2 functions in PHP are at least an order of magnitude faster so I'll be rewriting the resize functions with that in mind. GD2 wasn't an option when I first wrote the image gallery.

 

yup, socket 940 is the registered ram. expensive, but good.

Tip: G.Skill ram is a very, very solid brand that tends to sell for a little less than Corsair. Do you shop at newegg? That's the only place I buy anything for computers.

Ryan

 

I buy from Monarch computers. They know Linux pretty well, are local, great prices and good warrantees available. I don't like the thought of waiting on replacement parts via mail order if I have a problem. For the office systems I've purchased from Newegg and Zipzoomfly (formerly Googlegear).