Servers Down: So why is this happening?

The latest announcements from your Hala Team

Moderators: Arkon, Top Team

Post Reply
Themicles
Game Server Admin and Web Monkey
Posts: 313
Joined: Thu Dec 29, 2005 1:46 pm
Location: Michigan
Contact:

Servers Down: So why is this happening?

Post by Themicles »

A week, two weeks ago? Don't remember right now... tired and in a bad mood at this point...

Anyway, the last actual power outage we had seems to have damaged the UPS that serves as surge protection and battery backup for Eostre (Tairis), Poseidon (Hala/Ysgard) and Phoenix (Database).

The reason Phoenix and Poseidon have been dropping lately is apparently due to the UPS being unable to power all three servers during a switch to battery.

At this very moment, there's nothing I can do about it except hope DTE gets its shit together and we quit having problems with the local power grid. I'll be calling APC as soon as I calm down enough that I'm not rude to their customer service agents, but a replacement of this unit may mean we're completely without battery backup as I can't afford to allow them to put a hold on my bank account to send me a new one before I send them this one.

The servers are going to remain down for up to an hour while I verify that the UPS is definitely bad.
World Leader: Tairis'nàdur
Administrator: Confederation of Planes and Planets
Server Host & Web Monkey: Hala The Heroic Domain of Ysgard
Themicles
Game Server Admin and Web Monkey
Posts: 313
Joined: Thu Dec 29, 2005 1:46 pm
Location: Michigan
Contact:

Post by Themicles »

After further investigation and testing, it has been determined that the APC UPS unit itself should be fine. However, a self test run on the unit determined that the battery is bad and needs to be replaced. We wont know much more until we replace the battery, as it did pass the rest of it's self-tests. Without a good battery, I can't do manual tests, such as switching it to battery to see if all three servers stay running. The new battery will cost $89.99.

Unfortunately, we're currently behind on money in RL. We're currently only staying afloat by doing limited advances on Kessa's next check and with everything that's been breaking down on us lately, we can't seem to get ahead. We may have a chance at breaking the cycle when Kessa starts getting her first checks from her new job. But we have a LONG list of things that need to be done to the car now that she's driving 40 miles to work each way (80 miles a day!). Brakes, motor mounts, front suspension bushings, general tune-up, and the big unknown: transmission. Long story short, either the collar for fifth gear is shot, or the cross-over lever on the gear selector is shot and rebuilding these transmissions is difficult at best. Many shops wont do this transmission, and the ones that will tend to charge quite a bit.

Complicating things with the servers is that our local power grid has been inconsistent for the last few weeks. Zeus and the DSL modem are in one room that isn't on a battery backup, and with the APC's battery being bad, the servers in this room aren't immune to the brief power issues either.

Right now what we need to solve this issue reliably is a replacement battery for the APC, ideally an add-on battery pack for the APC, and a new UPS for the other room. The ideal APC UPS for that room at minimum would have up to a 400 watt power output. Why APC? I've used battery backups from other manufacturers and have had problems. With APC, I've not had any problems. Yes, the current battery is bad and needs replacing, but it is a month shy of the 3 year replacement schedule anyway. Seems like it did it's job to me.

We will be using an existing donation to handle the battery replacement. If you are interested in helping out and would like to see a goal to reach, here are the two items we're looking at based on the information in this post.
APC BE650G on Newegg (For Zeus and DSL modem)
APC BR24BP Battery Pack on Newegg (For BR1500 serving the game servers)
World Leader: Tairis'nàdur
Administrator: Confederation of Planes and Planets
Server Host & Web Monkey: Hala The Heroic Domain of Ysgard
Post Reply