Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google Log In with Steam Sign In
  • Create Account
Photo

Memory issues

- - - - -

  • Please log in to reply
13 replies to this topic

#1
ONLINE   Dragonji

Dragonji

    Advanced Member

  • Members
  • PipPipPipPip
  • 514 posts
142
On the road to fame
  • LocationPoland

Recently, I've found out that rcon does not work if the server is up for about a month. I did some investigation and discovered this most probably happens due to memory consumption of silEnT server. I tested a bit, run test server with Omni-bot (so there's someone who play) and the rest you can see below.

 

thb.png

 

Is it possible it's a mod issue or should it be reported to OB dev team?


Edited by Dragonji, 18 August 2013 - 11:11 PM.


#2
OFFLINE   gaoesa

gaoesa

    Advanced Member

  • Management
  • PipPipPipPip
  • 4391 posts
341
Will become famous
  • LocationFinland

You can get more detailed information of what is using the memory with following commands:

 

pmap -x <process id>

 

and

 

cat /proc/<process id>/smaps

 

Take a snapshot and after some time when the memory consumption has risen significantly, another snapshot.

 

Also, there are engine related reasons why you should restart the server more often then one month interval. The server time will rotate for example.



#3
OFFLINE   Sol

Sol

    Regular Member

  • Coders
  • PipPipPip
  • 113 posts
28
On a distinguished road
  • LocationPoland

Yeah, The engine must be restarted every 28 days otherwise time will be bugged. 



#4
OFFLINE   hellreturn

hellreturn

    Management

  • Management
  • PipPipPipPip
  • 2693 posts
158
On the road to fame

Memory leaks come from OB. I also get those. More bots you run more memory leak you get. I had reported that to crapshoot but seems like he is busy ...cos he used to reply pretty fast before. 



#5
OFFLINE   crapshoot

crapshoot

    Newbie

  • Members
  • Pip
  • 4 posts
0
Unknown

I don't remember any memory leak information being reported to me. Please send it. One thing that you can do is set SleepBots to 1 in the omni_bot.cfg which will disable bots while there are no humans on the server; reducing cpu cycles and memory usage during those times. Once a human joins, the bots will resume from where they were.



#6
ONLINE   Dragonji

Dragonji

    Advanced Member

  • Members
  • PipPipPipPip
  • 514 posts
142
On the road to fame
  • LocationPoland
pmap of a server which was running for 14 days:
30367:   ./etded.x86 +set fs_basepath /xxx/xxx/et_test +set fs_homepath /xxx/xxx/et_test +set vm_game 0 +set sv_pure 1 +set dedicated 2 +set fs_game silent +set sv_punkbuster 0 +set com_hunkmegs 56 +exec server.cfg +set net_port 50100 +set rconpassword test
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000008048000     408     376       0 r-x--  etded.x86
00000000080ae000      20      20      16 rwx--  etded.x86
00000000080b3000    8000     872     856 rwx--    [ anon ]
0000000008996000  149892  149888  149888 rwx--    [ anon ]
00000000ee340000    3576    3436    2288 r-x--  omnibot_et.so
00000000ee6be000      12      12      12 rwx--  omnibot_et.so
00000000ee6c1000    2948    2744    2744 rwx--    [ anon ]
00000000eecd8000      84      56       0 r-x--  libpthread-2.13.so
00000000eeced000       4       4       4 r-x--  libpthread-2.13.so
00000000eecee000       4       4       4 rwx--  libpthread-2.13.so
00000000eecef000       8       4       4 rwx--    [ anon ]
00000000eecf1000      28      12       0 r-x--  librt-2.13.so
00000000eecf8000       4       4       4 r-x--  librt-2.13.so
00000000eecf9000       4       4       4 rwx--  librt-2.13.so
00000000eedc8000    2308    2308    2308 rwx--    [ anon ]
00000000ef009000     112      12       0 r-x--  libgcc_s.so.1
00000000ef025000       4       4       4 rwx--  libgcc_s.so.1
00000000ef026000     896     504       0 r-x--  libstdc++.so.6.0.17
00000000ef106000      16      16      16 r-x--  libstdc++.so.6.0.17
00000000ef10a000       4       4       4 rwx--  libstdc++.so.6.0.17
00000000ef10b000      28      12      12 rwx--    [ anon ]
00000000ef112000    1492    1340     112 r-x--  qagame.mp.i386.so
00000000ef287000      52      52      52 rwx--  qagame.mp.i386.so
00000000ef294000   51236   46484   46484 rwx--    [ anon ]
00000000f249d000      64      48       0 r-x--  libresolv-2.13.so
00000000f24ad000       4       4       4 r-x--  libresolv-2.13.so
00000000f24ae000       4       4       4 rwx--  libresolv-2.13.so
00000000f24af000       8       4       4 rwx--    [ anon ]
00000000f24b1000      16      16       0 r-x--  libnss_dns-2.13.so
00000000f24b5000       4       4       4 r-x--  libnss_dns-2.13.so
00000000f24b6000       4       4       4 rwx--  libnss_dns-2.13.so
00000000f24b7000      40      20       0 r-x--  libnss_files-2.13.so
00000000f24c1000       4       4       4 r-x--  libnss_files-2.13.so
00000000f24c2000       4       4       4 rwx--  libnss_files-2.13.so
00000000f24c3000      36      16       0 r-x--  libnss_nis-2.13.so
00000000f24cc000       4       4       4 r-x--  libnss_nis-2.13.so
00000000f24cd000       4       4       4 rwx--  libnss_nis-2.13.so
00000000f24ce000      76      20       0 r-x--  libnsl-2.13.so
00000000f24e1000       4       4       4 r-x--  libnsl-2.13.so
00000000f24e2000       4       4       4 rwx--  libnsl-2.13.so
00000000f24e3000       8       0       0 rwx--    [ anon ]
00000000f24e5000      24      16       0 r-x--  libnss_compat-2.13.so
00000000f24eb000       4       4       4 r-x--  libnss_compat-2.13.so
00000000f24ec000       4       4       4 rwx--  libnss_compat-2.13.so
00000000f24f7000   82560   51532   51532 rwx--    [ anon ]
00000000f7597000    1392     652       0 r-x--  libc-2.13.so
00000000f76f3000       8       8       8 r-x--  libc-2.13.so
00000000f76f5000       4       4       4 rwx--  libc-2.13.so
00000000f76f6000      12      12      12 rwx--    [ anon ]
00000000f76f9000     144      72       0 r-x--  libm-2.13.so
00000000f771d000       4       4       4 r-x--  libm-2.13.so
00000000f771e000       4       4       4 rwx--  libm-2.13.so
00000000f771f000       8       8       0 r-x--  libdl-2.13.so
00000000f7721000       4       4       4 r-x--  libdl-2.13.so
00000000f7722000       4       4       4 rwx--  libdl-2.13.so
00000000f7723000       8       8       8 rwx--    [ anon ]
00000000f7725000      52      52      52 rwx--    [ anon ]
00000000f7732000       4       4       0 r-x--    [ anon ]
00000000f7733000     112     104       0 r-x--  ld-2.13.so
00000000f774f000       4       4       4 r-x--  ld-2.13.so
00000000f7750000       4       4       4 rwx--  ld-2.13.so
00000000ffb50000     516     496     496 rwx--    [ stack ]
----------------  ------  ------  ------
total kB          306304  261336  257004
pmap of a server with uptime of 5 min:
17750:   ./etded.x86 +set fs_basepath /xxx/xxx/et_test +set fs_homepath /xxx/xxx/et_test +set vm_game 0 +set sv_pure 1 +set dedicated 2 +set fs_game silent +set sv_punkbuster 0 +set com_hunkmegs 56 +exec server.cfg +set net_port 50100 +set rconpassword test
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000008048000     408     376       0 r-x--  etded.x86
00000000080ae000      20      20      16 rwx--  etded.x86
00000000080b3000    8000     832     768 rwx--    [ anon ]
0000000009e77000    9360    9336    9336 rwx--    [ anon ]
00000000ee311000    3576    3452    2288 r-x--  omnibot_et.so
00000000ee68f000      12      12      12 rwx--  omnibot_et.so
00000000ee692000    2948    2744    2744 rwx--    [ anon ]
00000000eeca9000      84      52       0 r-x--  libpthread-2.13.so
00000000eecbe000       4       4       4 r-x--  libpthread-2.13.so
00000000eecbf000       4       4       4 rwx--  libpthread-2.13.so
00000000eecc0000       8       4       4 rwx--    [ anon ]
00000000eecc2000      28      12       0 r-x--  librt-2.13.so
00000000eecc9000       4       4       4 r-x--  librt-2.13.so
00000000eecca000       4       4       4 rwx--  librt-2.13.so
00000000eee71000    1560    1560    1560 rwx--    [ anon ]
00000000eeff7000     112      12       0 r-x--  libgcc_s.so.1
00000000ef013000       4       4       4 rwx--  libgcc_s.so.1
00000000ef014000     896     508       0 r-x--  libstdc++.so.6.0.17
00000000ef0f4000      16      16      16 r-x--  libstdc++.so.6.0.17
00000000ef0f8000       4       4       4 rwx--  libstdc++.so.6.0.17
00000000ef0f9000      28      12      12 rwx--    [ anon ]
00000000ef100000    1492    1324     112 r-x--  qagame.mp.i386.so
00000000ef275000      52      52      52 rwx--  qagame.mp.i386.so
00000000ef282000   51236   40912   40880 rwx--    [ anon ]
00000000f248b000      64      48       0 r-x--  libresolv-2.13.so
00000000f249b000       4       4       4 r-x--  libresolv-2.13.so
00000000f249c000       4       4       4 rwx--  libresolv-2.13.so
00000000f249d000       8       4       4 rwx--    [ anon ]
00000000f249f000      16      16       0 r-x--  libnss_dns-2.13.so
00000000f24a3000       4       4       4 r-x--  libnss_dns-2.13.so
00000000f24a4000       4       4       4 rwx--  libnss_dns-2.13.so
00000000f24a5000      40      20       0 r-x--  libnss_files-2.13.so
00000000f24af000       4       4       4 r-x--  libnss_files-2.13.so
00000000f24b0000       4       4       4 rwx--  libnss_files-2.13.so
00000000f24b1000      36      16       0 r-x--  libnss_nis-2.13.so
00000000f24ba000       4       4       4 r-x--  libnss_nis-2.13.so
00000000f24bb000       4       4       4 rwx--  libnss_nis-2.13.so
00000000f24bc000      76      20       0 r-x--  libnsl-2.13.so
00000000f24cf000       4       4       4 r-x--  libnsl-2.13.so
00000000f24d0000       4       4       4 rwx--  libnsl-2.13.so
00000000f24d1000       8       0       0 rwx--    [ anon ]
00000000f24d3000      24      16       0 r-x--  libnss_compat-2.13.so
00000000f24d9000       4       4       4 r-x--  libnss_compat-2.13.so
00000000f24da000       4       4       4 rwx--  libnss_compat-2.13.so
00000000f24e6000   82556   45884   45884 rwx--    [ anon ]
00000000f7585000    1392     640       0 r-x--  libc-2.13.so
00000000f76e1000       8       8       8 r-x--  libc-2.13.so
00000000f76e3000       4       4       4 rwx--  libc-2.13.so
00000000f76e4000      12      12      12 rwx--    [ anon ]
00000000f76e7000     144      72       0 r-x--  libm-2.13.so
00000000f770b000       4       4       4 r-x--  libm-2.13.so
00000000f770c000       4       4       4 rwx--  libm-2.13.so
00000000f770d000       8       8       0 r-x--  libdl-2.13.so
00000000f770f000       4       4       4 r-x--  libdl-2.13.so
00000000f7710000       4       4       4 rwx--  libdl-2.13.so
00000000f7711000       8       8       8 rwx--    [ anon ]
00000000f7713000      52      52      52 rwx--    [ anon ]
00000000f7720000       4       4       0 r-x--    [ anon ]
00000000f7721000     112     104       0 r-x--  ld-2.13.so
00000000f773d000       4       4       4 r-x--  ld-2.13.so
00000000f773e000       4       4       4 rwx--  ld-2.13.so
00000000ffe52000     480     468     468 rwx--    [ stack ]
----------------  ------  ------  ------
total kB          164984  108736  104336
Test server bin: official 2.60b

Edited by Dragonji, 02 September 2013 - 11:12 AM.


#7
OFFLINE   TheSilencerPL

TheSilencerPL

    Advanced Member

  • Management
  • PipPipPipPip
  • 3099 posts
216
On the road to fame
  • LocationPoland

Possible small leak in qagame.so?

This is not relevant, we shoud get more samples, for instance from 1h intervals. If the memory would grow constantly we could consider the memory leak.



#8
OFFLINE   gaoesa

gaoesa

    Advanced Member

  • Management
  • PipPipPipPip
  • 4391 posts
341
Will become famous
  • LocationFinland

I don't see significant differences in these regarding qagame given that memory usage differences can be caused by other factors as well. One small leak we have found though, which happens when reading files with Lua scripts.

00000000ef112000    1492    1340     112 r-x--  qagame.mp.i386.so
00000000ef287000      52      52      52 rwx--  qagame.mp.i386.so
00000000ef294000   51236   46484   46484 rwx--    [ anon ]
00000000ef100000    1492    1324     112 r-x--  qagame.mp.i386.so
00000000ef275000      52      52      52 rwx--  qagame.mp.i386.so
00000000ef282000   51236   40912   40880 rwx--    [ anon ]

The major concern seems to be with the etded.x86.

0000000008048000     408     376       0 r-x--  etded.x86
00000000080ae000      20      20      16 rwx--  etded.x86
00000000080b3000    8000     872     856 rwx--    [ anon ]
0000000008996000  149892  149888  149888 rwx--    [ anon ]
0000000008048000     408     376       0 r-x--  etded.x86
00000000080ae000      20      20      16 rwx--  etded.x86
00000000080b3000    8000     832     768 rwx--    [ anon ]
0000000009e77000    9360    9336    9336 rwx--    [ anon ]


#9
OFFLINE   TheSilencerPL

TheSilencerPL

    Advanced Member

  • Management
  • PipPipPipPip
  • 3099 posts
216
On the road to fame
  • LocationPoland

anon is probably a stack memory, so I guess we may not care about it so much. We would need more samples, as I said, to be certain about the results.



#10
OFFLINE   gaoesa

gaoesa

    Advanced Member

  • Management
  • PipPipPipPip
  • 4391 posts
341
Will become famous
  • LocationFinland

The anon blocs are mostly memory allocated from the heap.



#11
OFFLINE   TheSilencerPL

TheSilencerPL

    Advanced Member

  • Management
  • PipPipPipPip
  • 3099 posts
216
On the road to fame
  • LocationPoland

Well, anon blocks can be both, actually. For example thread stacks can be anon blocks. But let's not argue about it, as we cannot formulate any conclusions based on 2 samples only, in my opinion.



#12
OFFLINE   gaoesa

gaoesa

    Advanced Member

  • Management
  • PipPipPipPip
  • 4391 posts
341
Will become famous
  • LocationFinland

That is true and it is stated already for qagame. However, the 140 MB increase in the etded.x86 is significant enough to see there is an issue. Accompanied by the fact that it is already known before hand that there is a memory leak problem. These snapshots are, in my opinion, clear enough to point to the direction where the memory concerned in the topic is not being freed. That is why I posted about two snapshots instead of doing a cronjob task for it. The 6 MB with qagame can come from a number of naturally occurring differences and it is not enough to formulate conclusions.



#13
OFFLINE   TheSilencerPL

TheSilencerPL

    Advanced Member

  • Management
  • PipPipPipPip
  • 3099 posts
216
On the road to fame
  • LocationPoland

In fact, the misunderstanding comes from my busy head and the fact that I didn't notice the difference in etded.x86 as I was writing the post from work and didn't focus too much on your post. Sorry for that. The rest remains true of course :)



#14
OFFLINE   hellreturn

hellreturn

    Management

  • Management
  • PipPipPipPip
  • 2693 posts
158
On the road to fame

After 11 days of uptime:

Address           Kbytes     RSS   Dirty Mode   Mapping
0000000008048000     748     408       0 r-x--  etded.x86
0000000008103000      20      20      16 rwx--  etded.x86
0000000008108000    8096     848     848 rwx--    [ anon ]
0000000009a0d000  288584  288496  288496 rwx--    [ anon ]
00000000df9d4000    1500    1316     112 r-x--  qagame.mp.i386.so
00000000dfb4b000      52      52      52 rwx--  qagame.mp.i386.so
00000000dfb58000   37552   33484   33484 rwx--    [ anon ]
00000000e2004000      84      52       0 r-x--  libpthread-2.13.so
00000000e2019000       4       4       4 r-x--  libpthread-2.13.so
00000000e201a000       4       4       4 rwx--  libpthread-2.13.so
00000000e201b000       8       4       4 rwx--    [ anon ]
00000000e201d000      28      12       0 r-x--  librt-2.13.so
00000000e2024000       4       4       4 r-x--  librt-2.13.so
00000000e2025000       4       4       4 rwx--  librt-2.13.so
00000000e2279000     112      12       0 r-x--  libgcc_s.so.1
00000000e2295000       4       4       4 rwx--  libgcc_s.so.1
00000000e2296000     896     508       0 r-x--  libstdc++.so.6.0.17
00000000e2376000      16      16      16 r-x--  libstdc++.so.6.0.17
00000000e237a000       4       4       4 rwx--  libstdc++.so.6.0.17
00000000e237b000      28      12      12 rwx--    [ anon ]
00000000e3a9c000    3576    3452    2288 r-x--  omnibot_et.so
00000000e3e1a000      12      12      12 rwx--  omnibot_et.so
00000000e3e1d000    2948    2744    2744 rwx--    [ anon ]
00000000e4434000   25716   25716   25716 rwx--    [ anon ]
00000000e5d51000      64      48       0 r-x--  libresolv-2.13.so
00000000e5d61000       4       4       4 r-x--  libresolv-2.13.so
00000000e5d62000       4       4       4 rwx--  libresolv-2.13.so
00000000e5d63000       8       4       4 rwx--    [ anon ]
00000000e5d65000      16      16       0 r-x--  libnss_dns-2.13.so
00000000e5d69000       4       4       4 r-x--  libnss_dns-2.13.so
00000000e5d6a000       4       4       4 rwx--  libnss_dns-2.13.so
00000000e5d6b000      40      20       0 r-x--  libnss_files-2.13.so
00000000e5d75000       4       4       4 r-x--  libnss_files-2.13.so
00000000e5d76000       4       4       4 rwx--  libnss_files-2.13.so
00000000e5d77000      36      16       0 r-x--  libnss_nis-2.13.so
00000000e5d80000       4       4       4 r-x--  libnss_nis-2.13.so
00000000e5d81000       4       4       4 rwx--  libnss_nis-2.13.so
00000000e5d82000      76      20       0 r-x--  libnsl-2.13.so
00000000e5d95000       4       4       4 r-x--  libnsl-2.13.so
00000000e5d96000       4       4       4 rwx--  libnsl-2.13.so
00000000e5d97000       8       0       0 rwx--    [ anon ]
00000000e5d99000      24      16       0 r-x--  libnss_compat-2.13.so
00000000e5d9f000       4       4       4 r-x--  libnss_compat-2.13.so
00000000e5da0000       4       4       4 rwx--  libnss_compat-2.13.so
00000000e5da6000  287268  105908  105908 rwx--    [ anon ]
00000000f762f000    1392     640       0 r-x--  libc-2.13.so
00000000f778b000       8       8       8 r-x--  libc-2.13.so
00000000f778d000       4       4       4 rwx--  libc-2.13.so
00000000f778e000      12      12      12 rwx--    [ anon ]
00000000f7791000     144      68       0 r-x--  libm-2.13.so
00000000f77b5000       4       4       4 r-x--  libm-2.13.so
00000000f77b6000       4       4       4 rwx--  libm-2.13.so
00000000f77b7000       8       8       0 r-x--  libdl-2.13.so
00000000f77b9000       4       4       4 r-x--  libdl-2.13.so
00000000f77ba000       4       4       4 rwx--  libdl-2.13.so
00000000f77bb000       8       8       8 rwx--    [ anon ]
00000000f77bd000      32      32      32 rwx--    [ anon ]
00000000f77c5000       4       4       0 r-x--    [ anon ]
00000000f77c6000     112     108       0 r-x--  ld-2.13.so
00000000f77e2000       4       4       4 r-x--  ld-2.13.so
00000000f77e3000       4       4       4 rwx--  ld-2.13.so
00000000ff99a000     508     476     476 rwx--    [ stack ]
----------------  ------  ------  ------
total kB          659844  464676  460348






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users