iPhone 4 Pre-Sales Clog AT&T's IT Arteries

Apple and AT&T began taking pre-orders for the iPhone 4 in stores an online on Tuesday, triggering a rush that swamped their IT systems. AT&T has temporarily suspended pre-orders, and Apple's site now tells pre-order customers that their phones will be shipped by July 2, rather than the official launch date of June 24.

The Tuesday rush also reportedly caused a system breach at AT&T. Some users say that when they attempted to log into their accounts, they were rerouted to strangers' accounts and given access to their information.

Both AT&T and Apple have declined comment.

All Dressed Up and Nowhere to Go

So many people tried to pre-order the iPhone 4 from AT&T and Apple that their IT systems reportedly were jammed and their switchboards flooded.

The websites of both companies were either down or so busy that prospective customers had to endure long waits. Many of those who tried pre-ordering over the phone claimed they simply couldn't get through.

Jammed systems weren't the only source of frustration. Some customers who managed to log into their AT&T accounts to preorder an iPhone 4 were instead switched to other customer's accounts, according to Gizmodo. The blog listed several emails from readers complaining about this issue.

The problem was related to a major fraud update that went wrong, an unnamed AT&T insider reportedly told Gizmodo. That update was implemented on all of AT&T's systems over the weekend, shutting them down.

The source claimed AT&T did not test its system properly before iPhone 4 preorders launched.

Test Fast, Fail Fast, Adjust Fast - or Not

How could AT&T and Apple be caught flat-footed by the rush when they had gone through a similar experience with the launch of the original iPhone ... and the iPhone 3G, and the iPhone 3GS? In a conference call to discuss AT&T's second-quarter 2009 results, Rick Lindner, the carrier's chief financial officer, said AT&T's website had its "largest order day ever" on June 19, 2009, the day the iPhone 3GS was launched.

"I don't think there's any real excuse," Rob Enderle, principal analyst at the Enderle Group, told MacNewsWorld. "The systems simply don't appear to be working properly. They should have been fully tested to adequate loads, given this is far from the first iPhone event, and they have generally had heavy initial traffic for such events."

"It is extremely possible that a software update could cause such a problem," Randy Abrams, director of technical education at ESET, told MacNewsWorld. "Software updates have the potential to break things in unexpected ways. After performing an upgrade or update, the entire system needs to be viewed as untested and must be verified to work properly."

AT&T should have run automated test scripts that mimic millions of users logging in simultaneously and should also have run manual tests simultaneously with that, Abrams said.

"Sometimes, a human sees what the programming logic failed to anticipate," Abrams explained.

Other Possible Workarounds

Once AT&T and Apple saw the volume of orders that was coming in, they could have turned to a third-party company to temporarily supplement their IT systems.

"If they had planned correctly, they could have turned to HP and other companies that provide services to handle sudden peak loads," Enderle pointed out. "Or, you can use hosting companies and shift existing loads to them."

Telephone companies are held to extremely high reliability standards, so AT&T should have been prepared for sudden surges in demand, Enderle said.

Once More Into the Breach

Flawed logic in the programming of AT&T's system may have caused it to switch customers who logged into other customers' accounts, ESET's Abrams said.

"Something like this could have been caused by a single character in a line of code," Abrams pointed out. "If proper security is placed around the log-in process, then a system overload should simply deny a user access rather than randomly redirect him to an account he is not authorized to view."

Alternatively, the switchover might have been caused by a problem in Apple's systems.

"If AT&T's servers had to talk to Apple's servers when a customer logged in, and Apple had changed something in its data format, the results of the log-in may be unpredictable," Abrams explained.

"This could violate a number of privacy laws," Enderle pointed out. "Potentially, given AT&T's size and coverage, it could represent massive damage to AT&T customers in terms of potential identity theft, fraud and other issues related to the release of this information."

This is AT&T's second security breach involving an Apple product in two weeks. Last week, the email addresses and device IDs of more than 100,000 iPad owners were exposed by an organization known as "Goatse Security." AT&T has vowed to take legal action against the group, one member of which has reportedly been arrested on drug charges following a police search of his house.