Getting Rid of Lagggg, Part 2By Jorrit Rouwe on Mar 17, 2014 9:00:00 PM
In this follow-up to the original Getting Rid Of Lagggg article, Lead Game Tech Jorrit Rouwe discusses the steps Guerrilla is taking to reduce lag in Killzone Shadow Fall.
What have we done since the last blog post?
We’ve increased the time it takes before you can be matched outside your own region. This means that, for a popular warzone like ‘24 Player Team Deathmatch’, 99% of EU/US players, 96% of LATAM/Australian/Japanese players, and 92% of the Asian players will now get matched in their own region. Almost all of the remaining players end up in their second closest region.
With patch 1.10 we also rolled out a server update which, due to a bug, caused the lag to increase. We rolled back the change, which resulted in the lag returning to previous levels.
Average ‘kill lag’ over time (the time it takes between firing the last bullet and seeing the enemy drop down)
Curiously, we’re still seeing reports from players that the lag has become worse over time. We spent a lot of time looking at all the ‘ping’ and ‘kill lag’ data available to us to determine a cause, but were unable to detect anything out of the ordinary. Indeed, the average kill lag has been consistent since we started tracking it in the beginning of December 2013, as shown in the graph above. With regards to the perception of lag, however, we did discover that players find a game to be very ‘laggy’ if only one or two kills during a match have a high kill lag – even if the rest of the match runs smoothly.
This prompted us to run a little experiment. We defined a ‘laggy kill’ as a kill that takes more than 0.5 seconds to register. We then determined that, given the measured ‘ping’ across the globe, the lowest realistically attainable percentage of laggy kills during a match would be 2%. Finally, we compared this number to what we could measure.
To our surprise, we saw that the percentage was really dependent on the region: US-East showed 13% laggy kills, US-West 6%, and other regions around 4%. Moving some of our servers in the US-East region to a different data center resulted in the percentage of laggy kills on those servers decreasing by a whopping 10%. A similar move in the US-West region unfortunately didn’t yield any further improvements.
So why is this? The US has many different networks, and not all networks have ‘peering agreements’ (a direct connection between the two networks). This means that sometimes, packets have to take a detour to arrive at their destination. For some players in the US-East region, we would even see packets travel from the east coast to the west coast and back before finally arriving at our server. This obviously introduces a lot of undesired lag.
All servers in the US-East region have been moved to the new data center, so players in this region should see an improvement.
Wireless versus Wired
For players who experience even more frequent ‘laggy kills’, we’ve found that the quality of their wireless connection may be a contributing factor. In such instances, switching to a wired connection has proven to halve the percentage of laggy kills.
The reason for this is that radio interference causes data to get lost much more easily on a Wireless connection. When that data happens to be your headshot, it takes a little while before the data is considered lost and is resent. This adds a couple of tenths of a second, and when the interference lasts longer (because, for instance, your neighbors are streaming a movie on their wireless connection on the same Wi-Fi channel) this process can easily cause a kill lag of a couple of seconds.
What is coming with the next patch?
Connection Quality Screen
We have been working on a feature that tells you how good your connection is. You can look at this information in the options screen.
The out game connection quality screen.
As you can see we measure a number of aspects of your connection:
- Nearest Server: This shows the country where the closest server is located. In-game this changes to the ‘Current Server’ which may be a more distant one if no players were found on the ‘Nearest Server’.
- Round-trip Time: Time it takes for a packet to travel to the ‘Nearest Server’ and back. In game this measures the round trip time to the ‘Current Server’ so may give you a worse score.
- Packet Loss: The amount of packets lost in transit. A good connection should have less than 1%. When playing on Wi-Fi this percentage is usually higher than when playing Wired.
- Download: The measured download speed in kilobit/s.
- Upload: The measured upload speed in kilobit/s.
- Connection: If your connection is Wired or Wireless.
Each of these categories will get a quality score ranging from 0 to 5 bars. The lowest score is shown as the main ‘Connection Quality’ score and is also displayed on the score board and in the main multiplayer menu.
The main purpose of this connection quality screen is to allow you to determine if it is you who is lagging or if it is one or more of the other players in the match. It will also make it clear when you’re playing cross-region.
Server Ping Grouping
We believe that everyone should be able to play, but we understand that if you invested a lot of money in a good internet connection that you deserve to have a good game experience. In the next patch, matchmaking will try to group people with similar connection quality to the server together as much as possible.
Improved Party Matchmaking
We have improved party matchmaking. The matchmaking server will now assign the party to the region where most people come from. Previously the region would be determined by the party leader.
What are we working on?
In the future we will be optimizing the matchmaking system further for clan matches, so that a match between a clan from the US-East region and a clan from Japan may be hosted in the US-West region if that results in the lowest total ping for all players.
New server locations
We’re starting tests with a data center in Dallas to improve connection quality for players in the middle of the US. We’re also still investigating data centers in the Middle East, but unfortunately there are very few data centers there that meet our needs at this time. We are currently looking into a data center in Istanbul.
Return to Own Region
If you ended up playing cross-region because there were no games available in your own region, the game will check for game availability in your own region again upon round rollover. If a game has become available in your own region you will be returned to your own region.
We observed that most cross-region play occurs on custom Warzones where the player count is low. Currently we will only show you the amount of players that are in a custom Warzone, but in the future we will also display the amount of players that are playing in your own region. This will give you a better indication of the quality of the game.
Getting Rid of Lagggg, Part 2By Jorrit Rouwe
Getting rid of laggggBy Bart Suichies
Guerrilla Armory: VSA LS36 ShotgunBy Bart Suichies
Roland IJzermans InterviewBy Bart Suichies
Eric Boltjes Q&ABy Bart Suichies
Helghast Infantry Q&ABy Victor Zuylen
StA-61 RifleBy Victor Zuylen
Black Hand Q&ABy Victor Zuylen
StA-19 PistolBy Victor Zuylen
PNV-06 Petrusite CannonBy Victor Zuylen