Maybe there is a better term. A fake player is just like a real player as far as the server and other players can tell. It has no AI. It is just for testing. It exists only on the server. A lot of games have this feature (not claiming to have invented it or anything!).
I want to express my love for fake players. I just recently (yesterday) got this feature working in Zero Gear.
Here are some of the advantages to adding fake players to your multiplayer game:
- Makes testing bandwidth and other networking issues much easier. Even some gameplay code might only have a bug when there are X number of players on the server.
- Obviously a MMO would benefit from having fake players to test server load and whatever else those crazy MMO devs need to test.
- Profiling. Your game might work great with 6 people playing. What happens when there is a full server with 16? Physics might slow down, clients may not be able to render all those models, maybe too much data is being sent over the net. It is better to find these sort of issues before getting 16 real people together to test.
- Possible to translate the fake players into bots later. No promise of this for ZG! Since the fake player acts just like a real player, all it needs to do is send input to the server and the fake player is moving around.
The implementation was pretty easy after some code restructuring. I simply moved the client networking code (networking only, no graphics, input, sound, etc) into the server. Each fake player executes the same client networking code a real client executes and just like magic (Illusion, Michael), it works! This is because the server only really interacts with the real players through the network.
Overall I wish I would have gotten this feature in ZG a lot sooner! If you are developing a networked game, do it now!!!
11 comments:
What do these fake clients do? I mean do they drop packets and such therefore testing reliability as just sending data to a local port would not seem to offer much. Also if it has no AI then does it send data to the server or is it just a dumb client?
Any links to information or documents about the other games you say which use such a thing for testing?
I don't have any links. I know MMOs do this type of thing all the time.
They don't need to do anything. They exist only for testing. The important thing to note is the server and other players (real and fake) don't see the fake players any different from normal players. So if I am working on a team game mode for example, I can make 10 fake players join the server to test team assignment or auto team balance, etc. These fake players are real players as far as all the code is concerned.
They can also be used to test long term problems like memory fragmentation. Just having 10 players sit in a server over night will reveal quite a few problems.
Testing dropped packets and extra latency is in the networking layer. It isn't really related to fake players (although you could simulate some of the fake players as having a lot of packet loss in their connection to see what happens).
It would be trivial to add some code to make the fake player send random key presses so they drive around.
This technique might not be needed if you had a QA department full of testers. We don't, so it was easier to make fake testers ;)
So in essence these fake clients could actually just have been achieved by firing up a couple of client executables?
Although I think by them sending some data, as Ben Garney suggests in his GDC lecture( http://coderhump.com/austin08/ ), and viewing the logs would reveal alot of useful information.
Sure, you could run 16 client executables every time you want to test a feature. I find it easier to just type "SpawnFakePlayers(16)" into the server console. But yes, they act the same as running the client app.
Haha, I just watched Ben Garney's talk a few days ago, funny how that works out. He mentions leaving a bunch of clients running overnight in that talk. The fake players do the same thing but they live on the server, so you don't need more than 1 computer to run the test (I don't have 16 or even 4 computers lying around to run tests on).
The fake players send and receive data just like real players. They act as if a real player connected to the server and then went to bed. The inbound data from clients is minimal (and can be sent by randomly sending fake input state). What really is worth testing is the outbound data and for this, dumb fake players work really well. And they are really easy to implement (easier if you do it right at the start instead of a year into the project like I did).
Thanks for the discussion btw :)
So I know World of Warcraft they do not use fake clients... It just is much easier to test the code path with the actual clients themselves. I do know that in some cases the Fake Clients are useful but what I find is that people write code that ends up special casing the "Fake Client" and you end up never testing code paths correctly.
To solve this issue the company I work for created a way that allowed you to run one application, but would essentially create N clients as if they were actually coming from the client itself. The server did not even know the difference between the test client and real client, all code paths are then tested. It removes the need to special case code for fake clients and still get what we wanted. This even allowed an actual client to see the "Test Clients" as if they were really in the game.
There are several ways to accomplish this, one of which is abstracting out the core code into a DLL and allowing a parent application to subscribe to that DLL in such a way that would allow you to create a headless client, which could then just create N instances of the client code. And act as if it was the normal Zero Gear Client. If an actual Client was to join a game with the headless client he would not see anything different.
Andrew, your company's solution sounds nearly identical to my solution.
These fake players are seen by the server and other clients as real players. The only difference I can see is these fake players exist on the server instead of in a separate app in my case. I just open a socket connection to the server from within the server posing as a client.
All code paths are identical between real and fake players :)
I do not think you should have the client code on the server... there just is no reason for it. Plus it means you need to run a server in order to test the client code path and that seems like a huge overhead and its code duplication. I think you could accomplish the samething without have the overhead of a server and code duplication.
I agree that the full client should not be embedded into the server. The only client code I have in the server is the client networking code (I didn't make this too clear, fixed it in the post). It is basically one .cpp/.h. It just manages the connection.
Things like graphics, input, and sound (and other, client only code) are not included in the server.
I don't understand what you mean by run a server in order to test the client code path. The client is a separate app in my case. The client does need to connect to the server in order to test a lot of features however.
I updated the second to last paragraph in the post in order to explain my implementation better. I realize now that I wasn't entirely clear on how I handled fake players in ZG.
I should also mention that these fake players are almost free as far as resources or processing time go. Because it is just the networking code being executed on the server for these players (which isn't that much code, and is even partly threaded in my case), things like physics and graphics do not execute.
Of course, there is the overhead on the server of having another player connected, but that is the whole point. There is little extra overhead on top of that.
Awesome post! On the topic of testing via simulated input, I have recently come to love unit-testing-- at least 25% of my time is devoted to writing tests to compare performance of a subsystem based on changes to my algorithms/implementation. It's really gratifying when you can actually put a number to an optimization (example: pure asynchronous rendering with a fixed update heartbeat has recently improved the rendering performance of video/high-animation content in Awesomium by up to 40%!) and it helps keep track of performance regression too.
Google recently posted an interesting article related to this topic on their Chromium blog: Putting It to the Test
I was thinking about getting a few tests written using http://unittest-cpp.sourceforge.net/ at some point. What is your process for writing tests?
Post a Comment