How Home Office detects suspicious traffic & keeps passport services online 24/7

Passport applications are one of the U.K.’s most popular public services. Between 4 and 8 million applications are processed by His Majesty’s Passport Office (HMPO) every single year.

HMPO offers not only one of the most popular U.K. public services, but also one of the most digitally advanced—with a world-leading 92% of passport applications submitted online.

HMPO’s impressive digital transformation has been recognized in awards like:

• The Best Use of Data and Technology at the 2021 Civil Service Awards
• The Most Innovative Use of Data in the Cloud at the 2022 Cloud Excellence Awards
• The Rant & Rave Best Customer Satisfaction Strategy Award at the 2018 Institute of Customer Service’s annual awards

It’s a high-impact, high-value, high-visibility service. The HMPO team see their website as the “shop window” for the U.K.’s public sector and a model for what digital-first service design can achieve.

Discover how HMPO used Queue-it’s virtual waiting room to achieve “zero outages”, get the “visibility and flexibility to act” on suspicious traffic, and “stay up to date with customer expectations.”

The number of U.K. residents who travelled abroad in 2020 plummeted 74% year over year. And when people don’t travel, they don’t apply for passports.

“We had to prepare for a much higher set of demand ahead of lockdowns lifting,” says Elaine Salveta, HMPO’s Digital Services Manager. “A normal year would see 3-4 million passport applications. We planned for more than double that, expecting up to 9.8 million applications.”

“We could see a storm brewing,” says Kevin Lewis, HMPO’s Product Manager. “Computing resources are finite. Demand is infinite. We just couldn’t be 100% sure we’d be able to keep the service up and running.”

Adding to the pressure of these potentially massive application numbers, Salveta says, was the fact that “customer expectations really changed during COVID. Customers now expect a service that never goes down. They expect the online experience to be fully controlled and well-managed. And if there’s high demand, they expect a queue.”

Traffic surges aren’t a new problem for HMPO. For years, they’ve dealt with huge spikes in traffic every time the service is mentioned or linked to on the news.

“The passport service is massively sensitive to media attention, regardless of the situation,” says Simon Watson, Technical Lead at HMPO. “If an article appears on any news outlet about us, it can quickly pick up steam and go viral across the news. These outlets typically link to our site, so the traffic just exponentially snowballs from there.

"And it’s almost impossible to prepare for these kinds of spikes,” he continues. “You can’t model and configure a load test that accurately simulates your website being mentioned on the National BBC news channel, the BBC website, and a bunch of other news sites simultaneously.”

In the past, these media-induced traffic spikes had caused runs on the passport service that brought it crashing down. The HMPO team knew from these incidents the damaging domino effects of website crashes, and just how important it is to avoid them.

At large and critical government organizations like Home Office, a site crash is much more than just a site crash, the team explains.

“If the service is down, you won’t be able to take in applications, you won’t be able to make money, and it’ll look very unprofessional,” says Lewis. “And it’s not just about the digital flow, but also the physical flow of document handling and ensuring we don’t overwhelm the back office. That's the key thing for us, being able to consistently stay up to avoid both starving and flooding the business.”

If the Passport website crashes, Salveta explains, “It’s not just the technical team working on resolving the issue. We’ll also be communicating to directors and updating the minister where appropriate. We’ll have to keep our contact center up to date, because they’ll be getting swarmed with requests. We’d have to put out messaging that explains when we expect to go back online. We’d need to engage our reputation team, who handle press briefings and social media chatter.

"From a political point of view, too, we’re answerable to how our service performs. And we really feel the pressure of that,” Salveta says. “We have nowhere to hide. If we’re down for even a couple of minutes, someone will Tweet about it and then the news channels will jump on it and next thing you know it’s all over the headlines.”

Site crashes—whether caused by COVID traffic spikes or by news stories—threatened HMPO’s revenue, resources, and reputation. In short, they were an avoid-at-all-cost problem.

The HMPO team considered several methods for handling traffic peaks before implementing Queue-it. They’d tried acting fast to scale up on the cloud and had even built their own basic waiting room solution.

But neither of these were enough. The risks were too high and the costs too large.

“Paying for super scaled up infrastructure is pointless when we don't need it most of the time. Queue-it offered that extra level of security and the ability to control the traffic spikes. It’s a very cost-effective way of managing demand."

Kevin Lewis, Product Manager

Lewis and the technical team were confident a virtual waiting room would be valuable for improving the user journey and supporting their digital transformation. But they had to make the business case to HMPO leadership.

“We told them ‘If you want to be a digital organization, you need to think digitally first. Passport services are a major shop window for the U.K. Government, and you simply can’t have that service go down. It’s unprofessional and just not what customers expect.’ We showed them how major U.K. retailers were using Queue-it and the effect it was having there. And we basically said ‘We need to spend this money to make us a proper digital organization. What we’re paying for with Queue-it is safety and reliability.’ And that’s paid off 100 times over.”

With the leadership team on board, Queue-it was integrated into the HMPO tech stack via the AWS Cloudfront Connector—and HMPO hasn’t had an outage since.

“Passports is one of the biggest digital services in the U.K. and we don’t collapse under pressure,” says Lewis. “Since we’ve had Queue-it we haven’t even come close. To be able to say we’ve had zero outages in the face of what was at least triple the amount of demand that we would expect is amazing. It's a really, really positive thing.”

“If you go down, it's the worst thing in the world. Because it’s part of your profession, it’s the service you have stewardship for,” says Watson. “We’re held to having a worldwide 24-hour available service. And solutions like Queue-it help you ensure that’s maintained.”

“Queue-it really gives us and the business the confidence we need. When the CEO asks ‘are you ready for a surge in demand?’ We can confidently answer yes,” says Salveta. “Beyond COVID, we now know that we’re prepared for peaks in demand. We can handle those days when we get almost 70k applications. It’s all nice and stable, and we’re keeping the customer experience completely under control."

The first time HMPO’s traffic exceeded the inflow limit they’d set and customers were queued, the team was nervous—they weren’t sure how customers would react.

“We just didn’t know how people would respond,” says Lewis. “We were searching all over social media to see how people would react, but we saw no adverse reaction at all. We realized it’s just what users expect from a modern digital service—if it’s busy, it doesn’t fall over, you don’t see service unavailable, you see a queue.”

The lack of a reaction was no surprise to Salveta, who says, “The waiting room is quite a nice customer experience, because people know what’s happening and what to expect next. It also gives us an opportunity to talk to and prepare the customer. We did some user research to determine what to include on the page, so people are ready when they get to the front of the queue—things like make sure you have your photos and get your old passport ready. We’ve even done some personalization on our waiting room and added a cute little plane that runs along the progress bar.”

When asked what they’d miss most about not having Queue-it, the technical team responds without hesitation “the insights into the traffic.”

“I have Queue-it up on one monitor every day while I work,” Lewis says. “It lets me keep an eye on our traffic, see whether we’re queuing people, and understand the reason behind it.”

“Google Analytics isn’t really accurate,” he explains. “The real-time data is a bit delayed and the numbers are off because it relies on people accepting analytics cookies. Queue-it’s more accurate and works better in real-time.”

And it’s not just the technical team using Traffic Insights, Salveta says. The dashboards and data have been used to communicate trends and learnings across the organization. “Queue-it’s Traffic Insights and reporting lets us communicate technical insights to non-technical stakeholders. We’ve shown the graphs to operations, directors, security, ministers. It makes a clear, quick visual point and helps us communicate in an easy-to-understand way, like how much traffic spiked, when the queue kicked in, and how long people were waiting.”

In addition to aiding internal communication and knowledge-sharing, Traffic Insights has let the HMPO team rapidly detect and block suspicious traffic.

“When we first implemented Queue-it, we saw this big spike of 200 or so users coming in one minute and back out the next,” Lewis says. "At first we ignored it because it wasn’t bringing us down and wasn’t causing the queue to activate. But it started becoming more and more frequent. We dug around and figured out what it was, and we were able to isolate it—and we wouldn’t have been able to do that without Queue-it.

“Then just a couple weeks ago,” Lewis continues, “we had a mobile device in Hong Kong hitting us about 150 times a minute and we didn’t understand why. We used to get sudden spikes like that and wonder if we were on the news or what was going on. But with the new Queue-it features we can go in and investigate and act on it. We can see ‘Okay we have an aggressive IP in Hong Kong’, and we can make a call on whether to block it or not. Before Queue-it, we wouldn’t have even known what was going on.”

“Queue-it allows us to investigate traffic to work out where demand is coming from and whether it’s genuine. And if it isn’t, we can isolate it and put it in what I call IP jail. From a security perspective that’s crucial. It gives us the visibility and flexibility to act fast.”

Elaine Salveta, Digital Services Manager

The U.K. passport service is already among the most digitized public facing services in the U.K. and one of the most advanced passport services in the world. But the HMPO team have no intention of slowing down.

“We have a culture of continuous improvement.” Says Salveta. “We know that customer expectations don’t stand still, so neither do we. It’s a world-class service we strive to deliver. We don’t just compare ourselves to other U.K. government departments, we compare ourselves to the private sector.”

Currently, the HMPO team is planning to take the final physical step—customers physically sending their old passports to the office—out of the passport application process.

“That’s the last piece of digitization from a citizen perspective,” says Watson. “So we’re quite excited to get those last remnants of paper out of the process.”

At the heart of the team’s mission, Lewis explains, is the desire for “people in government to take digital seriously.”

“We want to help people understand that it’s not just a website, it’s the front door to the public service. The idea is to move from projects to products. To focus on the constant evolution of services to keep them up to date with people’s expectations.

"The internet is a constantly evolving environment. And in a constantly evolving environment you can’t just drop something and walk away. You've got to keep people around who can maintain that and keep it relevant, keep it working, keep it safe, secure and everything else.

"It’s clear that’s what Queue-it and the private sector are doing. But that idea is gradually becoming more accepted in the U.K. government, and we’re kind of the model for it.”