AIX Performance commands

root's picture

How to Determine the load of your system

By executing uptime, you get the load on your system. The load indicates also the amount of programs that are being executed at the same time.

gz@aix~# uptime
04:19PM up 262 days, 1:57, 3 users, load average: 12.26, 11.86, 8.77

How to monitor the utilization for all logical CPUs:


gz@aix~# sar -P ALL 1 4
AIX lpar1 3 5 0XXXXXXXXXXX    12/07/11

System configuration: lcpu=4 ent=0.50 mode=Uncapped 

15:19:45 cpu    %usr    %sys    %wio   %idle   physc   %entc
15:19:46  0       57      37       6       0    0.19    37.3
          1       32      22      12      35    0.07    14.9
          2        0      26       0      74    0.00     0.1
          3        0      32       0      68    0.00     0.1
          U        -       -      11      37    0.24    47.7
          -       26      17      15      42    0.26    52.3
15:19:47  0       53      41       6       0    0.18    35.1
          1       24      23       7      46    0.07    13.9
          2        0      27       0      73    0.00     0.1
          3        0      30       0      70    0.00     0.1
          U        -       -      12      39    0.25    50.9
          -       22      18      15      46    0.25    49.1
15:19:48  0       49      37      13       1    0.29    58.4
          1       17      65      10       9    0.19    38.2
          2        0      26       0      74    0.00     0.1
          3        0      34       0      66    0.00     0.1
          U        -       -       1       3    0.02     3.3
          -       35      46      12       7    0.48    96.7
15:19:49  0       53      42       5       1    0.17    33.9
          1       41      21      21      18    0.08    16.9
          2        0      28       0      72    0.00     0.1
          3        0      30       0      70    0.00     0.1
          U        -       -      14      35    0.25    49.1
          -       25      18      19      39    0.25    50.9

Average   0       52      39       8       0    0.21    41.2
          1       25      41      12      21    0.10    20.9
          2        0      27       0      73    0.00     0.1
          3        0      31       0      69    0.00     0.1
          U        -       -       9      29    0.19    37.8
          -       27      25      15      33    0.31    62.2

How to monitor the system wide processor utilization:


gz@aix~# lparstat 1 3  

System configuration: type=Shared mode=Uncapped smt=On lcpu=4 mem=4096 psize=10 ent=0.50 

%user  %sys  %wait  %idle physc %entc  lbusy  vcsw phint
----- ----- ------ ------ ----- ----- ------ ----- -----
 30.5  39.9    7.7   21.9  0.38  76.4   37.0  1550   286 
 27.3  39.7    6.7   26.2  0.36  72.5   40.5  1641   285 
 35.4  46.1    6.5   12.0  0.43  87.0   37.7  1849   369 

How to get a running show of Hypervisor calls:


(source)
gz@aix~# lparstat -H 5 10

Now, the calls:
cede - a "cede" happens when a virtual CPU with no useful work to do enters a wait state, and gives -- or cedes -- its CPU capacity to another VP until such time as useful work appears.
confer - this call allows a virtual processor to give its cycles to one -- or all -- of the other VPs in its LPAR.
prod - this call makes a specific virtual processor runnable.
enter - this call adds an entry into the Page Frame Table. (I'll tell you about the PFT in another discussion).
put_tce - enters the mapping of a single 4096 byte page into the specified TCE ("Translation Control Entry"; again I'll discuss TCEs in another post).
read - lists the contents of specific Page Table Entries.
remove - invalidates entries in the Page Table.
bulk_remove - invalidates up to four (4) entries in the Page Table.
get_ppp - this call returns the LPAR's performance parameters.
set_ppp - allows the LPAR to modify its entitled CPU capacity percentage and variable capacity weight within certain limits.
clear_ref - clears reference bits in the specific PTE from the LPAR's Page Frame Table.
protect - sets protection bits in a specific PTE.
eoi - calls an interrupt reset function.
ipi - generates an interprocessor interrupt.
cppr - sets a CPU's current interrupt priority.
migrate_dma - serializes the sending of logical LAN messages to allow for page migration.
send_logical_lan - sends a logical LAN message
add_logicl_lan_buf - adds receive buffers to the logical LAN receive buffer pool (correlate with ' netstat -v ' output in a VIO client).
xirr - reports on virtual interrupts.
purr - provides a count of the ticks that a shared resource uses per virtual CPU or SMT thread. PURR stands for "Processor Usage Resource Register" and is invoked in all manner of performance tools.

How to display micropartition statistics with topas:


gz@aix~# topas -L
Interval:    2    Logical Partition: lpar1              Wed Dec  7 15:21:00 2011
Psize:       -                Shared SMT  ON           Online Memory:   4096.0
Ent: 0.50                      Mode: UnCapped          Online Logical CPUs:   4
Partition CPU Utilization                              Online Virtual CPUs:   2
%usr %sys %wait %idle physc  %entc %lbusy    app    Vcsw    phint   %hypv   hcalls
  47   33     9    11   0.4  89.44  26.16      -    2492      411    55.9    40144
==================================================================================
LCPU  minpf majpf  intr    csw   icsw runq lpa  scalls usr sys _wt idl   pc   lcsw
Cpu0    292     0  1456   5072   1093    3 100    9643  52  41   6   1 0.24   1166
Cpu1    344     0   706   3622    849    0 100   11180  54  33   8   6 0.21   1285
Cpu2      0     0    10      0      0    0   0       0   0  26   0  74 0.00     20
Cpu3      0     0    11      0      0    0   0       0   0  29   0  71 0.00     20

How to display cpu utilization of all active partitions in the management system (in other words, the lpars from the same squadron):


gz@aix~# topas -C
Topas CEC Monitor             Interval:  10             Wed Dec  7 15:18:23 2011
Partitions Memory (GB)           Processors
Shr:  5    Mon:79.0  InUse:75.0  Shr:8.5  PSz: 10   Don: 0.0 Shr_PhysB  7.29
Ded:  0    Avl:   -              Ded:  0  APP:  1.3 Stl: 0.0 Ded_PhysB  0.00

Host         OS  M Mem InU Lp  Us Sy Wa Id  PhysB  Vcsw Ent  %EntC PhI
-------------------------------------shared-------------------------------------
lpar1        A53 U   28  27 12  89  8  0  2   4.70    0  4.00 117.6   0
lpar2        A53 U   16  14  8  84  9  0  6   1.55 3188  1.50 103.5 1206
lpar3        A53 U   12  12  6  88  5  0  5   0.72 1426  0.50 143.2 610
lpar4        A53 U   16  15  8   3 10  0 85   0.29 3338  1.50  19.1 399
lpar5        A53 U  8.0 7.8  4   0  1  0 97   0.03    0  1.00   3.4   0

Host         OS  M Mem InU Lp  Us Sy Wa Id  PhysB  Vcsw  %istl %bstl
------------------------------------dedicated-----------------------------------

How to collect and display performance statistics for all logical CPUs in the system:


In the example below, we will show one time the 20 seconds data.
gz@aix~# mpstat 20 1

System configuration: lcpu=2 ent=0.8 mode=Uncapped 

cpu  min  maj  mpc  int   cs  ics   rq  mig lpa sysc us sy wa id   pc  %ec  lcs
  0  301    0  160  450  556  262    0   22 100 3808 30 60  0 10 0.05  6.8  392
  1   17    0  160  220   59   41    0   23 100  290 14 48  0 38 0.01  0.8  228
  U    -    -    -    -    -    -    -    -   -    -  -  -  0 92 0.74 92.4    -
ALL  318    0  320  670  615  303    0   45 100 4098  2  4  0 93 0.06  7.6  620

How to list of processes sorted by swap usage:


gz@aix~# ps gv | head -n 1; ps gv | egrep -v "RSS" | sort +5n
to calculate total used:
gz@aix~# ps gv | egrep -v "RSS" | awk '{total+=$5} END{print total}'

How to list of processes sorted by memory usage:


gz@aix~# ps gv | head -n 1; ps gv | egrep -v "RSS" | sort +6n
to calculate total used:
gz@aix~# ps gv | egrep -v "RSS" | awk '{total+=$6} END{print total}'

How to artificially limit the amount of memory:


gz@aix~# rmss -c 3000 
Simulated memory size changed to 3000 Mb. 

How to determine the IO load on your systems disks with iostat:


If you get the message "Disk history since boot not available", you will have to activate iostat with /usr/sbin/chdev -l sys0 -a "iostat=true".
gz@aix~# iostat 5 2

tty: tin tout avg-cpu: % user % sys % idle % iowait
0.1 4.8 8.7 18.5 51.2 21.6

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk1 1.6 16.4 3.5 6873724 81062953
hdisk0 1.8 16.9 3.7 10202425 80417825
cd0 0.0 0.0 0.0 0 0

tty: tin tout avg-cpu: % user % sys % idle % iowait
0.0 135.2 17.4 82.6 0.0 0.0

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk1 0.0 0.0 0.0 0 0
hdisk0 2.4 15.2 3.4 76 0
cd0 0.0 0.0 0.0 0 0

Tool to check the memory and paging space is allocation:


gz@aix~# svmon (with -P you can see per process allocation)

How to determine the amount of asynchronous I/O (legacy) servers configured on your system:


gz@aix~# pstat -a | egrep ' aioserver' | wc -l
or
gz@aix~# ps -ek | grep aioserver | grep -v posix_aioserver | wc -l

How to display virtual memory statistics:


root@aix53# vmstat -w 1 10 

System configuration: lcpu=34 mem=30976MB ent=5.80

 kthr          memory                         page                       faults                 cpu          
------- --------------------- ------------------------------------ ------------------ -----------------------
  r   b        avm        fre    re    pi    po    fr     sr    cy    in     sy    cs us sy id wa    pc    ec
  0   0    7061457     115394     0     0     0     0      0     0  9595   1833 34264 10 11 77  2  1.27  21.9
  1   0    7061454     115397     0     0     0     0      0     0  9201   1709 33079  9 11 77  2  1.23  21.2
 16   0    7061463     115192     0     0     0     0      0     0 12316   9676 33034 11 19 69  1  1.80  31.0
  3   0    7061464     114619     0     0     0     0      0     0 10842   6207 33429 10 17 68  5  1.66  28.6
  2   0    7061638     114443     0     0     0     0      0     0  6906   4124 28329  7 13 69 12  1.23  21.2
  0   0    7061131     114942     0     2     0     0      0     0  5757   2354 23583  5  9 75 11  0.90  15.5
  5   0    7061613     114457     0     2     0     0      0     0  6230   7724 23951  6  9 77  8  0.96  16.6
  1   0    7062157     113913     0     0     0     0      0     0  6340   2021 24501  6  8 81  5  0.90  15.4
  0   0    7062334     113736     0     0     0     0      0     0  5897   2530 22285  5  7 84  4  0.80  13.7
  2   0    7062425     113645     0     0     0     0      0     0  7378   2931 28937  6  9 82  3  0.92  15.8

A script to get memory utilization:


gz@aix~# cat meminfo
#!/usr/bin/ksh
#
# Quick view of memory usage under AIX
#
 
USED=`svmon -G | head -2 | tail -1 | awk '{ print $3 }'`
USED=`expr $USED / 256`
TOTAL=`lsattr -El sys0 -a realmem | awk '{ print $2 }'`
TOTAL=`expr $TOTAL / 1000`
FREE=`expr $TOTAL - $USED`
 
echo "\nMemory Information"
echo "=================="
echo "total memory = $TOTAL MB"
echo "free memory  = $FREE MB"
echo "used memory  = $USED MB"
 
exit 0

AIX temperature sensors:


By executing this command, you can (on supported platform) get the fan speeds and temperature of your system:
gz@aix~# /usr/lpp/diagnostics/bin/uesensor -a
3 0 11 31 P1
9001 0 11 2100 F1
9001 1 11 2760 F2
9001 2 11 1890 F3
9001 3 11 1890 F4
9002 0 11 5129 P1
9002 1 11 3129 P1
9002 2 11 5129 P1
9002 3 11 12077 P1
9004 0 11 3 P3-V1
9004 1 11 3 P3-V2
9004 2 11 3 P3-V3

How to disable lightweight memory trace:


This AIX tuning can be done for benchmarking reasons. In a productive environment this should not be disabled unless asked by AIX support team in the process of resolving an issue.
# raso -r -o mtrc_enabled=0

How to disable component trace:


This AIX tuning can be done for benchmarking reasons. In a productive environment this should not be disabled unless asked by AIX support team in the process of resolving an issue.
# ctctrl -P memtraceoff

How to disable run-time error checking:


This AIX tuning can be done for benchmarking reasons. In a productive environment this should not be disabled unless asked by AIX support team in the process of resolving an issue.
# errctrl -P errcheckoff

How to disable storage keys:


This AIX tuning can be done for benchmarking reasons. In a productive environment this should not be disabled unless asked by AIX support team in the process of resolving an issue.
# skeyctl -k off

Thou shalt not steal!

If you want to use this information on your own website, please remember: by doing copy/paste entirely it is always stealing and you should be ashamed of yourself!
Have at least the decency to create your own text and comments and run the commands on your own servers and provide your output, not what I did!

Or at least link back to this website.