xp_bugfix - nwn2server bugfixes

[Skywing]

Added release 1.0.7.

This build adds a GameObjUpdateTime .ini option that you can use to change how long the server waits before recomputing game object update messages sent to players.

From profiling a CPU-intensive high-end PW [40+ concurrent players], I have found that a significant amount of server CPU time is often spent computing game object update related processing on servers with many players and many active game objects. Increasing the GameObjUpdateTime value will save on server CPU time by decreasing how often the server tries to compute game object updates. For this PW, increasing the GameObjUpdateTime to 400000 made the difference from ping times spiking to 500ms or more to having a smooth gameplay experience.

If your server is getting behind (lagging), you may actually improve responsiveness for your players by increasing the GameObjUpdateTime, as this ensures the server doesn't get bogged down trying to send game updates faster than it can process them.

The game uses a default GameObjUpdateTime of 200000 microseconds. You can easily double this to 400000 with relatively minimal player impact. (Players using the Client Extension will see even less impact from this change.)

It's best to experiment around a bit with the value to choose a right one for your server. If your server doesn't have performance problems, keeping the default value is probably fine. If your server often becomes very slow after it's been up for a little bit, try upping the GameObjUpdateTime to 400000. It is recommended that you keep the GameObjUpdateTime to under 1000000 microseconds in order to maintain a smooth gameplay experience.

(Combat messages are sent separately from GameObjUpdate messages in general and combat responsiveness is generally independent of this option. So is chat responsiveness.)

You can always reset the value to its default by commenting out or deleting the .ini directive.

[Skywing]

Added release 1.0.8 which adds a way to control the GameObjUpdateTime on the fly via nwscript.

You may use the following script fragment to do so:

[Skywing]

Release 1.0.9 posted.

This release improves networking performance. Be sure to install the updated AuroraServerNetLayer.zip (uncompress it in the directory where nwn2server.exe is) when upgrading to 1.0.9

Clients should load areas much faster than with the stock game (or earlier xp_bugfix versions) with 1.0.9. The degree of actual speedup depends on the number of non-static objects in the area as well as how fasts the disks on a particular client computer are.

Particularly when increasing the GameObjUpdate time, area loading performance should be far superior when using this release.

Server CPU load should be reduced for clients loading areas as well.

Additionally, the retransmit aggressiveness of the networking code has been increased, in an effort to help clients that get behind after dropping several packets. In these cases, clients should be able to "catch up" faster.

There is a new, optional configuration option: GameObjUpdateBurstSize = <burst size>

This controls the maximum amount of data that xp_bugfix allows the server to send in a single GameObjUpdate message, when the client's send queue is empty, or when the client is receiving the initial dynamic object update list after an area transition. The game's default is 400 bytes, and the xp_bugfix 1.0.9 default is 102400 bytes if you do not set the configuration option. Setting this value to 400 effectively turns off the GameObjUpdate bursting feature.

If xp_bugfix 1.0.9 is installed correctly, you should see the following log message in xp_bugfix.txt at startup:

2010-02-02 01:07:50 : * GameObjUpdate burst size: 102400 bytes (stock server default would be 400 bytes).

(Where 102400 is the GameObjUpdateBurstSize you have chosen, or 102400 if you have let the setting remain at its default.)

---

Technical details:

By default, the server will send non-static objects to clients on area load by sending a series of GameObjUpdate messages. The server attempts to limit the amount of data sent with each GameObjUpdate message to around 400 bytes, and by default, the server will send a GameObjUpdate message every 200 milliseconds.

This means that non-static objects are generally streamed to the server at a rate usually not exceeding ~2KBps.

A complex area might have 20-30KB worth of object data to send when a client loads an area, and with this data volume, the low data rate of the GameObjUpdate mechanism is a major component of area load times (presuming that client disks can quickly load appearance and other resource data for objects).

The release adds a 'bursting' mechanism that allows the first GameObjUpdate update to a client, after an area load, to exceed the usual 400 byte limit, up to the GameObjUpdateBurstSize. This typically improves throughput for clients (and also allows the server to better compress the object data, thus actually reducing the amount of data sent over the wire).

The bursting mechanism has been carefully constructed to only operate during area loading or when the client's frame send queue is nearly empty, so as to avoid severely damaging client performance when frames are lost on the network. (The game protocol does not support a very performant mechanism for dealing with lost packets when more than one packet is queued for transmission.)

[Skywing]

Release 1.0.10 has been posted.

This release adds a workaround for a server bug that causes clients to crash at logon in certain circumstances.

While the bug in question has been a long-standing server issue, the performance improvements in xp_bugfix 1.0.9 caused the issue to reproduce more frequently.

The reason why the problem happened more frequently with xp_bugfix 1.0.9 is that the server could now send the contents of non-static objects to clients in one message, instead of the updates being split over several messages with 200ms of delays in between each message.

The underlying problem is a server bug where, for the initial logon, the server erroneously begins sending object updates before it sends the area parameters to the client. If the server gets to the point of being able to update a certain class of objects on the client, before it sends the area parameters, then the client will crash.

Normally, should the server have hit the underlying bug, it would have been likely that the first GameObjUpdate would not update the type of object that would crash the client. The next update would happen after normal area loading had been triggered.

Because the server was now able to send the entire non-static object list in one message, the bug was no longer being papered-over in some cases by the low performance of object transmission.

Any users with the Client Extension active would have been protected from experiencing this bug. You can install xp_bugfix 1.0.10 to prevent users without the Client Extension active from experiencing this client crash bug.

[bsb5652]

Hello,

I was able to install the latest version ( 1.1.10 ) on my beta box that is a direct mirror environment of our prod box. I installed the VC Runtime installation for both 64 and 32 bit, and placed the dll and mdb in the installation folder of nwn2 as instructed on both servers. I also, created the ini file with the ReplaceNetLayer = 1 in it on both.

When I loaded on my production environment, I am getting this error in the application event log:

Log Name: Application
Source: SideBySide
Date: 2/13/2010 7:02:17 PM
Event ID: 33
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: ROT_MAIN_02
Description:
Activation context generation failed for "C:\Users\Administrator\Documents\Neverwinter Nights 2\xp_bugfix.dll". Dependent Assembly Microsoft.VC80.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="8.0.50727.4053" could not be found. Please use sxstrace.exe for detailed diagnosis.

I've tried everything and the server already has the .NET framwork 3.5 installed with all applicable service packs.

If you have any ideas, they would be greatly appreciated.
_________________
Brian S. Bloom
Creator of NWN2 PW Realms of Trinity

[PNJ]

Skywing ... you rock !

[bsb5652]

I ran windows update one more time and it picked up a .NET security update and now works perfect!
_________________
Brian S. Bloom
Creator of NWN2 PW Realms of Trinity

[PNJ]

Skywing, the lastest xp_bugfixes add a lots of improvement in areas loading. But if i rode right, it'is only working for area placeable ? if yes, could it be possible to apply the same improvement on character ?

DM loading are longer than player, and so i was wondering if that was due to the npcs to be always rendered by the DM (what ever their distance to him) that could cause this slow load

[Skywing]

Objects that are not affected by the change (and which were already sent 'quickly') include:

- Waypoints (i.e. map pins)
- Static sounds
- Static placeables
- Static cameras
- Environmentals
- Trees
- Static doors

All other objects are transmitted via the GameObjUpdate differencing mechanism.

If you are speaking of initial logon times for DMs, there is a large amount of data sent to the DM client relating to items that may be created in the creator. This data isn't sent by the GameObjUpdate mechanism although if the DM client and the server both use xp_bugfix, general network performance ought to be better (not to the same magnitude as removing the perf limitations in GameObjUpdate however).

[erikbreau]

Any chance someone would care to share on how to create the xp_bugfix.ini file that is needed to change the 'ReplaceNetLayer = 1' ?

Erik.
_________________
"Europa is coming, and there isn't anything you can do to stop it!..."
http://nwn-europa.blogspot.com/
Europa Pic Gallery: http://picasaweb.google.ca/erikbreau/EuropaPics

[Poleeko]

I am in a similar boat as erikbreau. I tried creating the xp_bugfix.ini as a text file, but not sure if it did any good.

[erikbreau]

I actually succeeded. It's simple once you know how...

open notepad.
create a notepad file entitled 'xp_bugfix.ini'
the content of this file should only be:

ReplaceNetLayer = 1

then save it to the NWNX4 directory folder... that's it!

Erik.
_________________
"Europa is coming, and there isn't anything you can do to stop it!..."
http://nwn-europa.blogspot.com/
Europa Pic Gallery: http://picasaweb.google.ca/erikbreau/EuropaPics

[PNJ]

Skywing, do you think it could be doable to make nwn2server.exe process to be multi threated ? i mean for improved performance.

[Palamor]

I just installed xp_bugfix for the first time. I don't think it worked, since the server still crashes (rather quickly) when players are chased by baddies.

I am running XP64. Should I have installed the 64-bit version of "Microsoft Visual C++ 2005 Service Pack 1 Redistributable Package ATL Security Update"? I did place the aurora support files in the nwn2.exe directory.

Are there any other microsoft updates I need? Help?

[pontificus]

I would like to point out something that confused me. You actually need more than just the ReplaceNetLayer =1 in your xp_bugfix.ini

This is mine:

DisableServerCompression = 1
ReplaceNetLayer = 1
GameObjectUpdateTime = 300000

I didn't fiddle with the other parameter... But then in my xp_bugfix.txt I can see that these are taking effect. For a while I only had ReplaceNetLayer, because thats what I read in this thread and my load times sucked majorly. I just made this change today so I'm not sure that its tweaked 100% right but I thought I'd mention it

Pontificus Maximus