The Rice Thresher

Location: http://the.ricethresher.org/news/2005/09/30/email_it_system

September 30, 2005 > News > Server outages disrupt e-mail

Server outages disrupt e-mail

Replacement system to be installed next semester

By Ian Everhart | Thresher staff

Undergraduates saw widespread e-mail problems last weekend, which prevented them from receiving e-mails sent to college listservs about hurricane preparations on campus.

IT Systems, Architecture and Infrastructure Director Barry Ribbeck said an overloaded Owlnet e-mail server caused students to receive multiple copies of e-mails, experience a lag in receiving e-mails and have e-mails not show up in inboxes. Owlnet e-mail account holders — all undergraduates and some graduate students — encountered these problems beginning Sept. 21. The problem was fixed Monday, Ribbeck said.

Diane Butler, the manager for IT’s client services, said College Computing Associates informed her that students were receiving duplicate e-mails late in the day Sept. 21.

“At that point, I simply told the CCAs that most of our staff was in the process of evacuating, that we may not even have mail if we lose power, and that we would deal with this first thing Monday morning,” Butler said.

Ribbeck said the e-mail outage illustrated the need for alternative means of communication with students. Khan said Rice’s emergency Web page was successful and could be used to reach students in the future. Ribbeck also said students should provide an external e-mail address to the Registrar’s Office in case Rice’s e-mail system is lost, like those at Tulane and Lamar universities were as a result of recent hurricanes.

“In the registrar’s system, they have a field for an external e-mail address, but it’s very incomplete,” Ribbeck said. “It’s not required by the Registrar’s Office, but since it’s there, people should fill it in. The problem is that students usually have free e-mail accounts, and they don’t guarantee you anything — they can time out, go inactive or fill up.”

Contributing problems

Ribbeck said IT’s diagnostic system had to be shut down when all non-essential systems went offline. Normally, the system warns IT personnel as the server nears its capacity.

“We need to restructure [emergency] monitoring so only critical systems are monitored,” Ribbeck said. “That would have told us the disks were getting full. Instead, we had to rely on calls coming into the help desk, which is usually after the problem has already occurred.”

Academic and Research Computing Director Rick Peterson said IT staff members were not aware of the extent of the problems until Monday morning, although staff members had worked on them remotely over the weekend.

Vice Provost for Information Technology Kamran Khan said keeping communication channels open for the students who stayed on campus during the hurricane was a priority.

“Four people were physically here — they slept [in Mudd Lab] — because we knew the students were here, and we wanted to make sure we were here for them,” Khan said. “The main thing was to keep the communications up as much as we could.”

Ribbeck said he and three other IT staff members came to campus several times per day to monitor the problem, and about 20 others worked remotely. Many IT staff members left Houston because they live in areas designated for mandatory evacuation by Houston Mayor Bill White and Harris County Judge Robert Eckels, Ribbeck said. Peterson said IT wanted to ensure staff members’ safety by allowing them to evacuate if they needed to do so.

Khan said increased traffic — primarily e-mails about the approaching hurricane — strained the Owlnet mail server. Khan said the outage might still have occurred without the hurricane, but that the increase in traffic made the mail server reach its capacity more quickly, and that the staff shortages prolonged the time it took to address the problems.

“If Rita hadn’t happened, we probably would have still had this problem, but maybe would have done a better job if we weren’t so consumed with Rita,” Khan said.

Ribbeck said the problem resulted from a few students’ accounts being wrongly configured, other students’ e-mail accounts exceeding their quota and a long queuing period for over-quota accounts. Previously, e-mail messages were stored for 100 days if the receiving account holder’s inbox was over its quota. Ribbeck said the time was reduced to about 5 days Monday.

“The issue with queuing messages is: How long do you want to delay the return message to the sender to let them know about the problem in communication?” Ribbeck said. “An e-mail message being read 100 days late is probably fairly useless.”

In addition, accounts for individuals no longer at Rice, such as 2005 graduates, took up about 20 percent of the server’s space, Ribbeck said. Dormant accounts are usually deleted in October, Khan said.

Wrongly configured accounts were reset, dormant accounts were removed and the queuing time was changed Monday morning and afternoon, which allowed mail to flow to current account holders by Monday evening, Ribbeck said.

Ribbeck said no messages were lost at any point.

The future

As part of an ongoing $22-million network project, IT will convert to the Cyrus mail server, which Ribbeck said is more stable and has more features. And new e-mail storage equipment will allow for larger e-mail quotas, Ribbeck said. He said he hopes a pilot of the new server will be in place by Thanksgiving and that the switch will be complete in the spring semester.

Amber Obermeyer contributed to this report.

End of article

Back to top