This post was last updated on June 27, 2022 by Pauline De Polonia, Engineering Manager at Hostari.
On June 20, 2022, Patches 0.209.8 and 0.209.9 were released by Iron Gate Studio, causing some updated servers to lose world progress. Our engineering team received 80-90 tickets reporting this issue in the next few days following the update. The Hostari team mobilized all departments to come up with a resolution. Here's how we did it.
Root Cause
Updating Valheim version to 0.209.10 wipes out progress and causes the world to reset.
Upon further investigation, our engineering team found:
- a newly generated "/worlds_local" folder from which the worlds were being read, thus making it appear as if the worlds were "reset";
- the "/worlds" folder is still present and intact with the actual world progress.
Thus, in attempting to fix the world-loading issue from Patch 0.209.8, the most recent update failed to load the players' worlds from the "/worlds" directory.
Why the folder infrastructure was updated
Based on Patch notes 0.209.8, a steam cloud save bug causes players to "lose progress if they were playing on multiple accounts on the same PC." To fix this, the worlds_local folder was generated, which contained a newly spawned world.
“Local files will now be stored in “worlds_local” and “characters_local” under AppData and no longer be synced to cloud to avoid sync conflicts and dataloss when using multiple accounts on the same machine, and when using dedicated servers.”
Resolution
Our engineering team had saved backup files of both the "/worlds" and "/worlds_folder" and sent them over to the customers for them to upload and verify. We exerted all efforts in restoring the world progress following Iron Gate Studio's advice from its official Valheim Discord channel to remove broken files from "/worlds_local" and move the files from the "/worlds" folder to "/worlds_local."
Limitations
- Modded servers cannot be updated to the latest game version as the mods are not yet compatible with the latest version. Our solution was to remove the mods and then restore the progress of the worlds once a customer approves playing on a non-modded server in the meantime.
- As much as our engineering team tried restoring their worlds, some customers reported that their worlds have regressed to a few days' back progress. According to the patch notes, the "steam cloud sync" functionality added to the game was buggy upon release.
These changes are quite unprecedented from a game studio, and we at are just as disappointed at this release. We will continue to work with our customers to restore worlds to the best of our abilities, but please know that losing progression is through no fault of our own—other hosters are also experiencing similar problems. Patch 0.209.10 should be the latest stable release.
Moving Forward
Although the latest patch has stirred up turmoil within the server folder infrastructure, The Hostari team was able to keep the chaos at bay. After the patch notes were released, our engineering team monitored updates on the official Valheim Discord and came up with a solution for the world progress loss. We were lucky timing-wise since the team was already building a more stable infrastructure that reduces common issues such as backups not saving properly. As this was happening, we also kept track of ongoing and incoming tickets with the help of the marketing and product teams.
Improvements:
- Our "Update Valheim Version" button on the server page can now handle migrating world files from worlds to worlds_local folders, reducing the time it takes for a manual fix (it takes around 7 or less minutes to update the servers manually; with automation, it is down to a minute).
- Infrastructure update improves our backup system.
- Mods currently tested and supported after the update: ValheimPlus, EquipmentAndQuickSlots, UseEquipmentInWater, EpicLoot, ServerSideMap, MonsterLabZ, Valheim Legends, OdinShip, KnarrTheTrader, RunicPack, TeleportEverything, Jewelcrafting, Blacksmiths Tools, HookGenPatcher, EpicValheimsAdditions, PlanBuild - Blueprint Rune, Supply Crates, EpicMMOSystem, ChaosArmor, BoneAppetit, Spawn That, DoOrDieMonsters, Valharvest, Do Or Die Biomes, Wildlife, Fee Fi Fo Fum, Fantasy Armoury, Do or Die Items, Upgrade World, Marketplace And Server NPCs Revamped, Farmyard Animals, Shipyard, Fantasy Creatures, Fantasy Creatures Config, Deds Army, Food Barrel, Potion Shelf, CraftyCartsRemake, ServerCharacters, Official BepInEx ConfigurationManager, Minotaurs, TargetPortal, Jotunn, Fancy Foods, Auga, Fruit Trees, Infinity Hammer, Stronghold, Woodshed, Mining, RRRCore, Custom Raids, DaedricBow, RecipeCustomizer, Judes Equipment, This Goes Here, SpeedyPaths, jtv biomes, Drop That, Valheim Plus, ChickenBoo, Sages Vault, WeaponCustomizer, PopVillages, AutoFeed, AnyPortal, ColorfulSigns, Terraheim, JsonDotNet, TrashItems, RRRNpcs, RRRMonsters, Surtling_Boat.
Our team is also watching for mod updates and testing for full functionality. We will send updates from time to time about which mods are compatible with the latest version. We also advise customers to back up their worlds frequently, and we plan to add automated backups in the future based on a schedule.
