I am placing this thread here specifically to discuss the AI sim speed slowdown that people are experiencing on water maps.

Please keep the discussion on topic so we can hopefully get this issue resolved.

Please post your own experiences with sim speed slowdowns. Please post as much detail as possible such as map, number of AIs, which AIs, any mods that are installed, and how long into the game the slowdown started.

Hopefully the experienced AI modders on this forum will be able to find a solution.

I'll start by summarising things which I think are known/proved.

1. It's not hardware/system related!
2. Water maps in general are the worst sim-speed wise but sim speed can become low on land maps. Setons appears to be the worst map of all.
3. The issue is not related to presence of water units.
4. Teamed AIs have a negative sim-speed impact versus one AI of the same size (unit count).
5. Simple water maps such as the lakes are not affected as badly.
6. Red herrings: Increasing game speed and dropping it later has no effect. TotalVeterancy has no effect.
7. Killing AIs/units etc has little positive impact on sim-speed. Once it slows down, it stays slowed down.
8. Duncane fixes have definately improved performance. Generally making large land maps playable. (Stopping units stuck in base?)
9. Very large/long games with AIs tend to become unreliable. Increased chance of clients hanging/crashing etc.

Lets think of a standardised test which we can conduct. Perhaps a saved game, or a set of settings that someone can use to determine how well SupCom runs on their system, and how well FA runs on their system.

I recommend we decide on a set of 3 maps of varying sizes. Setons should be one, because its a large map which includes water. I recommend we also decide on the AI to use - perhaps we should not use DuncanE's and Sorian's AIs, because they arent stock AI. As great as they are, what we are trying to find out is, if the standard user installed FA, would his experience with the AI be slower than his system specs would suggest. Sorian and DuncanE may both have valuable insight into what is causing the slowdown, however.

I propose that the test is as follows: A user hosts a skirmish game, with himself set to observe (is this possible in skirmish?). He zooms all the way out, and leaves the camera like that. This isnt an FPS test, its a simspeed test, so we should keep everyone's graphics as similar as possible. I recommend we use 4 AI players, and in some games have them allied, in some games not. Then we measure the simspeed at various intervals. Then repeat the test in SupCom (thus maps must be available for both) and see what the difference is.

Bleugh - testing like that is going to take ages, but it might be the only way to conclusively prove that FA simspeed is lower than SupComs.

Stock AIs on Setons is going to prove nothing other than FA AI is broken. What you suggest sounds useful but more setting up of a decent land-based game which WOULD run just fine. And seeing if someone can run that okay. They should be able to, if not - there's something wrong at their end.

Its a pity there is no way to script the running of FA, and make it not display any graphics (ie run in a console window).

if there were, I would run simulations here, cos at work I have a dual quad core machine with 3GB of ram.

Why do we need to test against vanila? i really dont care about its performance, all i want to solve is the FA slowdown.


Sorian we need to divorce the slowdown from the 'water map' issue. the extreme slowdowns can happen in land maps as well. (the water map issue seems to exacerbate the slowdown issue though)

One thing i think we can all agree on is that setons should be the testing map, it is the most commonly cited map that experiences slowdown.

Additionaly i think we should all start playing with cheats on by default. when the slowdowns occur we will be able to test out several possibilities ie, turn off the AIs - observe results, destruct units - observe results,
destroy players etc.

and lastly i believe it would be useful to have three set tests, one with no human player, one with human vs AI, and one with human + AI allies vs AI.

it would be helpful if everyone could zip and upload replays showing the problem, and if you can post your CPU size and when in the gametime the slowdown occured.

Because lots of people believe its caused by people with inadequate system specs, or running antivirus, or whatever. Basically they believe its the fault of the system, and not FA.

What I hoped to achieve there was to show that most computers have no problems with Vanilla, but have problems with FA, and thus the problem lies with FA, not the system.

@AI-Modders

I skimmed over LUADOC (this one in particular cuz im at work: http://supcom.wikia.com/wiki/LUADOC_1.5.3596) and there doesn't seem to be much AI related..

<Sim.CAiAttackerImpl> ... seems to be attack-related stuff for every unit.
<Sim.CAiNavigatorImpl> ... seems to be pathfinding for every unit
<Sim.CAiPersonality> and <Sim.CAiBrain> seem to be mostly functions to retrieve information.

Therefore i guess the opponent-AI is completely done in LUA or did i miss something?

Honestly if you are posting in this thread we know its not your system. i think we have seen too many threads derailed into calling the users system at fault. when we all know the sort of performance a quad core or high end dual core should be capable of.

This thread should be all about the sim speed, the only caveat being that it would help if you post your CPU when describing your replay and the sim speeds experienced.

okay first replay.

http://www.users.on.net/~frost/slowdown-8ai.scfareplay

mods:
-latest sorian AI

CPU: Quad @ 3Ghz

8 AI 4v4 on setons.

Speed.
10 mins +2
12 mins +0
15 min +0/-1
20 min -3
30 mins -7

At this point i take control of the enemy AIs one at a time (setfocusarmy), destroying all buildings and units and kill off two AI players.

play goes up very slowly to -5/-6

Kill off all but two opposing AI SCU's. (no units/buildings left)

Play is still at -5

give up, i cant make the speed go any higher.

of note is that even though i toggled off the AI (Alt+A) the commanders would still try to build and do their own stuff. but i had to toggle off the AI to destruct the units because the AI would stop the auto-destruct (Ctrl+K).



This may be a poor example because the 8 Ai would destroy any game, but i think its useful to illistrate that once the speed dives, the dropping units count or AI players will not bring it back up. (unlike a MP game)

i think this leads in the direction that unit count is not the real problem.

there are things an AI runs that cant be destroyed or stopped with ctrl - k maybee we should try admiral zeech's lead make the AI as light as possible and see how it runs.

Well, engineer pathing is not the issue. I thought since it was the only time that the AI regularly used CanPathTo it might be an issue, guess not. It also isn't the LowMass and LowEnergy functions (that don't work anyway).

Ran test on Seton's - 1v1 Sorian AIs.

After about 15 min it was at -2. A few minutes later I was at -4.

Same AIs on Strip Mine around the same time it was still +2 and stayed there until I stopped the game a while later.

Its also doesnt appear to be the AttackForceAi or GuardMarker pathing either. I just set all the land platoons to use HuntAi (eg no lua pathing) and the performance was the same on setons.

Basically the test I run is one UEF TechAi vs one Cybran RushAi.

On Setons this is at -2 by about 15 mins.
On Emerald crater it stays at at least +3 for 30+ mins.

I also dont think is related to the water on setons. I tried a test on another land bridge map - cant remember its name right now.. it was added for FA, is dark, 10km and similar landbridge layout to setons only smaller - dark heart maybe [EDIT] It was Haven Reef [EDIT]. On this map sim speed didnt drop below +2 even to the end of the game... maybe 40mins or so.

Its interesting to note that Setons is a very unusual map. It is largish (20Km) has 8 player spots and lots of mass (both spots and reclaimable). Also the default location for player 1 and 2 is close to each other. The AI's basically have one whole side of the map each to expand into and get mass. This means that on setons the AI produces lots of expansions, engineers and units quite quickly. Dare I say it but it may just be the map layout itself that leads the AI to a bad performance level on setons.

One other possibility is the EngineerManager code. Its seems to produce a lot of threads. Its possible these that arnt getting cleaned up. I havent thought of a good way to test this yet.

And a final interesting point to note ... I tested one EasyAi vs NormalAi on setons and I didnt see sim go below +3. The NormalAI won in about 20mins.

okay another game tonight.

1v1 Ai on setons (sorian 0.93)

10mins +7 speed
15mins +1
20mins -1

unit count 518/571

25mins -3 speed

unit count 606/682

kill all units

unit count 305/415

30mins -2 speed

kill everything but commanders (unit count is around 20/20 because of unfinished buildings)

31mins +0 speed.

so two units /AI's left in the game and it cannot rise above 0 speed.





is it possible that wrecks could be a issue? or the fact the game is keeping a record of what scouted buildings they think exist (even though they have been destroyed ?)

actually my earlier 8AI game didnt improve after killing off players so its probably not that.


could be the wrecks though.

i dont think, but they could be burdening reclaim functions....

but i am more in favor of duncanes explanation




some threads are not being killed when they should or when the associated platoon are destroyed. this could really add up and in the end cause slowdown, killing all units but commanders wouldn t not be of much relief for sim speed since in fact on the game side a lot of threads would be running even though the units are no more. the sim speed could be the result of ghost platoons being still piloted by the Ai.

I decided to try out the "20 mins no rush" option against the AI a couple of days ago. I noticed some interesting things AI related, one of them being the exacerbation of the sim speed slow down after the initial 10-15 min. The interesting thing is that on map where usually there was no slow down (Doroza's Sanctuary), I was able to force the game in a slow down state using the no rush option.

Another effect was the inability of the AI to do much expanding after the initial 20 mins was up. This inability to do much base expanding affected the Turtle AI a lot more then the Rush AI. This may all be unrelated to the current problem of sim slow down, but I though I'd share my experience hoping it'd give you guys some ideas.

Its worth pointing out that when you ctrl-K units it will take a while for the relevant threads to finish. The buildmanager threads for example run on a 13 second loop, the HuntAi on a 17 second loop. So you really need to leave it for a minute or so after destroying all the units to see if the sim speed improves.

Having said that, when I do wait I still only see a 1 or 2 point increase in sim speed. It never returns to the previous +10 max.

So either its losing threads somewhere or the sim always gets slower as the game gets longer and more units are involved even if wrecked (possible this is due to the way it calculates and passes the sim hash or whatever between players?)

If I get a chance tonight Im going to put together a version of AI that doesnt build expansion bases to see what kind of effect that has on setons (the ai builds A LOT of them on setons).

I thought it might be wreckage too. Would it be straight forward to mod out wreckage entirely to test that? Well, it generically might be reclaimable stuff. Setons is already rammed with trees.

GPG, are you reading this thread? can you give us some indication that you are looking into this problem? can you throw us a bone as it were?

It's not wreckage as such. You can play on a water map with islands, like gentleman's reef, and pretty much no units get destroyed and it rapidly slows to -3 or so.

One AI had built 30 T1 engies. So I destroyed them, thinking maybe this would point to some sort of hunt/reclaim type threads. Waited a minute. Still -3, hadn't budged.

The more I look at this, the more I think something very weird is going on under the hood. Is it not possible to somehow profile the LUA?

now, cartographic mode runs at a much higher framerate - but if you want to turn-off the graphics then try this console command:


this will toggle the world on or off. i think the result is different between vanilla and FA, in FA only the hud is rendered on a black background and possibly in vanilla the hud doesn't render either.

that should remove all graphics related bottle-necks, but it still would consume some resources and processor time.

is there any way of tracking how much memory the Sim, Lua, and UI sides consume? when i play, i notice most of my slow-downs when i run out of RAM and my computer is forced to shuffle data back and forth from the pagefile. for the record, i run on a 2.0GHz Athlon X2 with 1GB DDR2-800 RAM.

Okay... I havent got any closer to finding the exact cause but I have seen some performance improvements.

I put together an AI that doesnt build expansion bases on Setons and also doesnt use the GuardMarker AI routine.

You can try it out here:
http://members.iinet.net.au/~dionysus/misc/SetonsAI.zip

Remove any other AI's first (particular my AI fix pack.. its probably okay with Sorians installed). Select either the rush or adaptive AI on setons and it will use a custom AI plan just for that map. Best in a 1v1 situation (vs a player or AIvsAI)

It has problems (it doesnt take all its available mass spots), but it does seem to remain in positive sim speed for the game.

Let me know how you go.

A couple of other points in regards to what others have said.

I think we all know this, but its worth stating... its not gfx related. I have a 8800 GT and my frame rates never drop below 50 or so even when the sim is negative.

I believe it is possible to profile lua code (http://www.keplerproject.org/luaprofiler/manual.html) but it would have to be enabled/compiled into the SC lua engine and we dont have access to that code. Im sure GPG has a profiler tool for SC/FA that they have run or could use.

If there is anything i can help you guys out with let me know. i have a really high specced machine, and have this issue. It appears on setons only as well.

Unit limit appears to affect the sim speed massivley on setons.

as in i can play me+1 sorian cheating ai, vs 2 sorian cheating ai's with a 1000 unit limit, and only get normal slowdown. If i play with 8 ais' and a 500 unit limit, cheating ai's again i totally crawl in about half the time.

lowering graphics settings doesn't help the sim speed for me, but does prolong the crash. this was on sorain 0.88.
i havn't tried the newest version yet.