GitHub pull request #1795 of commit cea8588ce92f3a0867909087e38b944e42d8e02b, no merge conflicts.
Running as SYSTEM
Setting status of cea8588ce92f3a0867909087e38b944e42d8e02b to PENDING with url http://ci.skydive.network:8080/job/skydive-scale-tests/3248/ and message: 'Build started for merge commit.'
Using context: skydive-scale-tests
Setting status of cea8588ce92f3a0867909087e38b944e42d8e02b to PENDING with url http://ci-logs.skydive.network/skydive-scale-tests/builds/3248/log.html and message: 'Build started for merge commit.'
Using context: skydive-scale-tests
[EnvInject] - Loading node environment variables.
[EnvInject] - Preparing an environment for the build.
[EnvInject] - Keeping Jenkins system variables.
[EnvInject] - Keeping Jenkins build variables.
[EnvInject] - Injecting as environment variables the properties content 
MINIKUBE_DRIVER=virtualbox

[EnvInject] - Variables injected successfully.
[EnvInject] - Injecting contributions.
Building remotely on ha:////4NNp1pXCbYTTkmu3Cx99DP0KzgThOVpjNT8rxrgs1f0QAAAApR+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAz+EgZJ/eT83ILSktQi/azUvOzMvOL44pzEslRdIwBCsRomzgAAAA==jenkins_slave-2 (slaves vm) in workspace /var/lib/jenkins/workspace/skydive-scale-tests
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/skydive-project/skydive
 > git init /var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive # timeout=10
Fetching upstream changes from https://github.com/skydive-project/skydive
 > git --version # timeout=10
 > git fetch --tags --force --progress https://github.com/skydive-project/skydive +refs/heads/*:refs/remotes/origin/* # timeout=20
 > git config remote.origin.url https://github.com/skydive-project/skydive # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/skydive-project/skydive # timeout=10
Fetching upstream changes from https://github.com/skydive-project/skydive
 > git fetch --tags --force --progress https://github.com/skydive-project/skydive +refs/pull/*:refs/remotes/origin/pr/* # timeout=20
 > git rev-parse refs/remotes/origin/pr/1795/merge^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/pr/1795/merge^{commit} # timeout=10
JENKINS-19022: warning: possible memory leak due to Git plugin usage; see: https://wiki.jenkins-ci.org/display/JENKINS/Remove+Git+Plugin+BuildsByBranch+BuildData
Checking out Revision 5bebd28d8830bc6e18cb70a9c057570598d8d803 (refs/remotes/origin/pr/1795/merge)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 5bebd28d8830bc6e18cb70a9c057570598d8d803 # timeout=20
Commit message: "Merge cea8588ce92f3a0867909087e38b944e42d8e02b into c19cdb8a56303c76212c34f76106e19677c9203d"
 > git rev-list --no-walk 408c8ff13788ecc5059163a2a6a3edb7d3e98cff # timeout=10
First time build. Skipping changelog.
[skydive-scale-tests] $ /bin/bash /tmp/jenkins6832177653861144298.sh
cni0
docker0
erspan0
eth0
gre0
gretap0
lo
lxdbr0
virbr0-nic
gobuild-cache
VOLUME
/usr/bin/minikube
Deleting local Kubernetes cluster...
Errors occurred deleting machine:  Error deleting host: minikube: Error loading host from store: Host does not exist: "minikube"
Deleted Images:
untagged: busybox:latest
untagged: busybox@sha256:4b6ad3a68d34da29bf7c8ccb5d355ba8b4babcad1f99798204e7abb43e54ee3d
deleted: sha256:64f5d945efcc0f39ab11b3cd4ba403cc9fefe1fa3613123ca016cf3708e8cafb
deleted: sha256:d1156b98822dccbb924b4e5fe16465a7ecac8bfc81d726177bed403a8e70c972

Total reclaimed space: 1.199MB
53f3fd8007f76bd23bf663ad5f5009c8941f63828ae458cef584b5f85dc0a7bf
which: no skydive in (/sbin:/usr/sbin:/usr/bin:/bin:/usr/local/sbin:/usr/local/bin)
readlink: missing operand
Try 'readlink --help' for more information.
+ check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop == start ']'
+ '[' stop == stop ']'
+ '[' -z 10 ']'
+ stop 10 10 10
+ ANALYZER_NUM=10
+ AGENT_NUM=10
+ VM_NUM=10
+ echo 'Stopping Agents...'
Stopping Agents...
++ seq 10
+ for A in $( seq $AGENT_NUM )
+ stop_agent 1
+ AGENT=1
+ NAME=agent-1
+ '[' -e /tmp/skydive-scale/agent-1.pid ']'
+ sudo pkill -f agent-1.yml
+ timeout=30
+ pkill -0 -f agent-1.yml
+ sudo rm /tmp/skydive-scale/agent-1.lock /tmp/skydive-scale/agent-1.pid
rm: cannot remove '/tmp/skydive-scale/agent-1.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-1.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 2
+ AGENT=2
+ NAME=agent-2
+ '[' -e /tmp/skydive-scale/agent-2.pid ']'
+ sudo pkill -f agent-2.yml
+ timeout=30
+ pkill -0 -f agent-2.yml
+ sudo rm /tmp/skydive-scale/agent-2.lock /tmp/skydive-scale/agent-2.pid
rm: cannot remove '/tmp/skydive-scale/agent-2.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-2.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 3
+ AGENT=3
+ NAME=agent-3
+ '[' -e /tmp/skydive-scale/agent-3.pid ']'
+ sudo pkill -f agent-3.yml
+ timeout=30
+ pkill -0 -f agent-3.yml
+ sudo rm /tmp/skydive-scale/agent-3.lock /tmp/skydive-scale/agent-3.pid
rm: cannot remove '/tmp/skydive-scale/agent-3.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-3.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 4
+ AGENT=4
+ NAME=agent-4
+ '[' -e /tmp/skydive-scale/agent-4.pid ']'
+ sudo pkill -f agent-4.yml
+ timeout=30
+ pkill -0 -f agent-4.yml
+ sudo rm /tmp/skydive-scale/agent-4.lock /tmp/skydive-scale/agent-4.pid
rm: cannot remove '/tmp/skydive-scale/agent-4.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-4.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 5
+ AGENT=5
+ NAME=agent-5
+ '[' -e /tmp/skydive-scale/agent-5.pid ']'
+ sudo pkill -f agent-5.yml
+ timeout=30
+ pkill -0 -f agent-5.yml
+ sudo rm /tmp/skydive-scale/agent-5.lock /tmp/skydive-scale/agent-5.pid
rm: cannot remove '/tmp/skydive-scale/agent-5.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-5.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 6
+ AGENT=6
+ NAME=agent-6
+ '[' -e /tmp/skydive-scale/agent-6.pid ']'
+ sudo pkill -f agent-6.yml
+ timeout=30
+ pkill -0 -f agent-6.yml
+ sudo rm /tmp/skydive-scale/agent-6.lock /tmp/skydive-scale/agent-6.pid
rm: cannot remove '/tmp/skydive-scale/agent-6.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-6.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 7
+ AGENT=7
+ NAME=agent-7
+ '[' -e /tmp/skydive-scale/agent-7.pid ']'
+ sudo pkill -f agent-7.yml
+ timeout=30
+ pkill -0 -f agent-7.yml
+ sudo rm /tmp/skydive-scale/agent-7.lock /tmp/skydive-scale/agent-7.pid
rm: cannot remove '/tmp/skydive-scale/agent-7.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-7.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 8
+ AGENT=8
+ NAME=agent-8
+ '[' -e /tmp/skydive-scale/agent-8.pid ']'
+ sudo pkill -f agent-8.yml
+ timeout=30
+ pkill -0 -f agent-8.yml
+ sudo rm /tmp/skydive-scale/agent-8.lock /tmp/skydive-scale/agent-8.pid
rm: cannot remove '/tmp/skydive-scale/agent-8.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-8.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 9
+ AGENT=9
+ NAME=agent-9
+ '[' -e /tmp/skydive-scale/agent-9.pid ']'
+ sudo pkill -f agent-9.yml
+ timeout=30
+ pkill -0 -f agent-9.yml
+ sudo rm /tmp/skydive-scale/agent-9.lock /tmp/skydive-scale/agent-9.pid
rm: cannot remove '/tmp/skydive-scale/agent-9.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-9.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 10
+ AGENT=10
+ NAME=agent-10
+ '[' -e /tmp/skydive-scale/agent-10.pid ']'
+ sudo pkill -f agent-10.yml
+ timeout=30
+ pkill -0 -f agent-10.yml
+ sudo rm /tmp/skydive-scale/agent-10.lock /tmp/skydive-scale/agent-10.pid
rm: cannot remove '/tmp/skydive-scale/agent-10.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-10.pid': No such file or directory
+ echo 'Stopping Analyzers...'
Stopping Analyzers...
++ seq 10
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 1
+ ANALYZER=1
+ NAME=analyzer-1
+ '[' -e /tmp/skydive-scale/analyzer-1.pid ']'
+ sudo pkill -f analyzer-1.yml
+ timeout=30
+ pkill -0 -f analyzer-1.yml
+ sudo rm /tmp/skydive-scale/analyzer-1.lock /tmp/skydive-scale/analyzer-1.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-1.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-1.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 2
+ ANALYZER=2
+ NAME=analyzer-2
+ '[' -e /tmp/skydive-scale/analyzer-2.pid ']'
+ sudo pkill -f analyzer-2.yml
+ timeout=30
+ pkill -0 -f analyzer-2.yml
+ sudo rm /tmp/skydive-scale/analyzer-2.lock /tmp/skydive-scale/analyzer-2.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-2.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-2.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 3
+ ANALYZER=3
+ NAME=analyzer-3
+ '[' -e /tmp/skydive-scale/analyzer-3.pid ']'
+ sudo pkill -f analyzer-3.yml
+ timeout=30
+ pkill -0 -f analyzer-3.yml
+ sudo rm /tmp/skydive-scale/analyzer-3.lock /tmp/skydive-scale/analyzer-3.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-3.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-3.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 4
+ ANALYZER=4
+ NAME=analyzer-4
+ '[' -e /tmp/skydive-scale/analyzer-4.pid ']'
+ sudo pkill -f analyzer-4.yml
+ timeout=30
+ pkill -0 -f analyzer-4.yml
+ sudo rm /tmp/skydive-scale/analyzer-4.lock /tmp/skydive-scale/analyzer-4.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-4.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-4.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 5
+ ANALYZER=5
+ NAME=analyzer-5
+ '[' -e /tmp/skydive-scale/analyzer-5.pid ']'
+ sudo pkill -f analyzer-5.yml
+ timeout=30
+ pkill -0 -f analyzer-5.yml
+ sudo rm /tmp/skydive-scale/analyzer-5.lock /tmp/skydive-scale/analyzer-5.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-5.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-5.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 6
+ ANALYZER=6
+ NAME=analyzer-6
+ '[' -e /tmp/skydive-scale/analyzer-6.pid ']'
+ sudo pkill -f analyzer-6.yml
+ timeout=30
+ pkill -0 -f analyzer-6.yml
+ sudo rm /tmp/skydive-scale/analyzer-6.lock /tmp/skydive-scale/analyzer-6.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-6.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-6.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 7
+ ANALYZER=7
+ NAME=analyzer-7
+ '[' -e /tmp/skydive-scale/analyzer-7.pid ']'
+ sudo pkill -f analyzer-7.yml
+ timeout=30
+ pkill -0 -f analyzer-7.yml
+ sudo rm /tmp/skydive-scale/analyzer-7.lock /tmp/skydive-scale/analyzer-7.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-7.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-7.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 8
+ ANALYZER=8
+ NAME=analyzer-8
+ '[' -e /tmp/skydive-scale/analyzer-8.pid ']'
+ sudo pkill -f analyzer-8.yml
+ timeout=30
+ pkill -0 -f analyzer-8.yml
+ sudo rm /tmp/skydive-scale/analyzer-8.lock /tmp/skydive-scale/analyzer-8.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-8.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-8.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 9
+ ANALYZER=9
+ NAME=analyzer-9
+ '[' -e /tmp/skydive-scale/analyzer-9.pid ']'
+ sudo pkill -f analyzer-9.yml
+ timeout=30
+ pkill -0 -f analyzer-9.yml
+ sudo rm /tmp/skydive-scale/analyzer-9.lock /tmp/skydive-scale/analyzer-9.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-9.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-9.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 10
+ ANALYZER=10
+ NAME=analyzer-10
+ '[' -e /tmp/skydive-scale/analyzer-10.pid ']'
+ sudo pkill -f analyzer-10.yml
+ timeout=30
+ pkill -0 -f analyzer-10.yml
+ sudo rm /tmp/skydive-scale/analyzer-10.lock /tmp/skydive-scale/analyzer-10.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-10.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-10.pid': No such file or directory
+ sleep 5
+ sudo -E screen -S skydive-stress -X quit
No screen session found.
+ rm /tmp/skydive-scale/screen.lock
rm: cannot remove '/tmp/skydive-scale/screen.lock': No such file or directory
++ seq 10
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 1 10
+ IDX=1
+ VM_NUM=10
+ NAME=agent-1
+ echo Delete agent agent-1
Delete agent agent-1
++ cat /tmp/skydive-scale/agent-1-vswitchd.pid
cat: /tmp/skydive-scale/agent-1-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-1-dbserver.pid
cat: /tmp/skydive-scale/agent-1-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm1
Cannot remove namespace file "/var/run/netns/agent-1-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm2
Cannot remove namespace file "/var/run/netns/agent-1-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm3
Cannot remove namespace file "/var/run/netns/agent-1-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm4
Cannot remove namespace file "/var/run/netns/agent-1-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm5
Cannot remove namespace file "/var/run/netns/agent-1-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm6
Cannot remove namespace file "/var/run/netns/agent-1-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm7
Cannot remove namespace file "/var/run/netns/agent-1-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm8
Cannot remove namespace file "/var/run/netns/agent-1-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm9
Cannot remove namespace file "/var/run/netns/agent-1-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm10
Cannot remove namespace file "/var/run/netns/agent-1-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-1.lock
rm: cannot remove '/tmp/skydive-scale/agent-1.lock': No such file or directory
+ delete_host agent-1
+ NAME=agent-1
+ echo Delete Host agent-1
Delete Host agent-1
+ sudo ovs-vsctl del-port br-central agent-1-eth0
ovs-vsctl: no port named agent-1-eth0
+ sudo ip link del agent-1-eth0
Cannot find device "agent-1-eth0"
+ sudo ip netns del agent-1
Cannot remove namespace file "/var/run/netns/agent-1": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 2 10
+ IDX=2
+ VM_NUM=10
+ NAME=agent-2
+ echo Delete agent agent-2
Delete agent agent-2
++ cat /tmp/skydive-scale/agent-2-vswitchd.pid
cat: /tmp/skydive-scale/agent-2-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-2-dbserver.pid
cat: /tmp/skydive-scale/agent-2-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm1
Cannot remove namespace file "/var/run/netns/agent-2-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm2
Cannot remove namespace file "/var/run/netns/agent-2-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm3
Cannot remove namespace file "/var/run/netns/agent-2-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm4
Cannot remove namespace file "/var/run/netns/agent-2-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm5
Cannot remove namespace file "/var/run/netns/agent-2-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm6
Cannot remove namespace file "/var/run/netns/agent-2-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm7
Cannot remove namespace file "/var/run/netns/agent-2-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm8
Cannot remove namespace file "/var/run/netns/agent-2-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm9
Cannot remove namespace file "/var/run/netns/agent-2-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm10
Cannot remove namespace file "/var/run/netns/agent-2-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-2.lock
rm: cannot remove '/tmp/skydive-scale/agent-2.lock': No such file or directory
+ delete_host agent-2
+ NAME=agent-2
+ echo Delete Host agent-2
Delete Host agent-2
+ sudo ovs-vsctl del-port br-central agent-2-eth0
ovs-vsctl: no port named agent-2-eth0
+ sudo ip link del agent-2-eth0
Cannot find device "agent-2-eth0"
+ sudo ip netns del agent-2
Cannot remove namespace file "/var/run/netns/agent-2": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 3 10
+ IDX=3
+ VM_NUM=10
+ NAME=agent-3
+ echo Delete agent agent-3
Delete agent agent-3
++ cat /tmp/skydive-scale/agent-3-vswitchd.pid
cat: /tmp/skydive-scale/agent-3-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-3-dbserver.pid
cat: /tmp/skydive-scale/agent-3-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm1
Cannot remove namespace file "/var/run/netns/agent-3-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm2
Cannot remove namespace file "/var/run/netns/agent-3-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm3
Cannot remove namespace file "/var/run/netns/agent-3-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm4
Cannot remove namespace file "/var/run/netns/agent-3-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm5
Cannot remove namespace file "/var/run/netns/agent-3-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm6
Cannot remove namespace file "/var/run/netns/agent-3-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm7
Cannot remove namespace file "/var/run/netns/agent-3-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm8
Cannot remove namespace file "/var/run/netns/agent-3-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm9
Cannot remove namespace file "/var/run/netns/agent-3-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm10
Cannot remove namespace file "/var/run/netns/agent-3-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-3.lock
rm: cannot remove '/tmp/skydive-scale/agent-3.lock': No such file or directory
+ delete_host agent-3
+ NAME=agent-3
+ echo Delete Host agent-3
Delete Host agent-3
+ sudo ovs-vsctl del-port br-central agent-3-eth0
ovs-vsctl: no port named agent-3-eth0
+ sudo ip link del agent-3-eth0
Cannot find device "agent-3-eth0"
+ sudo ip netns del agent-3
Cannot remove namespace file "/var/run/netns/agent-3": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 4 10
+ IDX=4
+ VM_NUM=10
+ NAME=agent-4
+ echo Delete agent agent-4
Delete agent agent-4
++ cat /tmp/skydive-scale/agent-4-vswitchd.pid
cat: /tmp/skydive-scale/agent-4-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-4-dbserver.pid
cat: /tmp/skydive-scale/agent-4-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm1
Cannot remove namespace file "/var/run/netns/agent-4-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm2
Cannot remove namespace file "/var/run/netns/agent-4-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm3
Cannot remove namespace file "/var/run/netns/agent-4-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm4
Cannot remove namespace file "/var/run/netns/agent-4-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm5
Cannot remove namespace file "/var/run/netns/agent-4-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm6
Cannot remove namespace file "/var/run/netns/agent-4-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm7
Cannot remove namespace file "/var/run/netns/agent-4-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm8
Cannot remove namespace file "/var/run/netns/agent-4-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm9
Cannot remove namespace file "/var/run/netns/agent-4-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm10
Cannot remove namespace file "/var/run/netns/agent-4-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-4.lock
rm: cannot remove '/tmp/skydive-scale/agent-4.lock': No such file or directory
+ delete_host agent-4
+ NAME=agent-4
+ echo Delete Host agent-4
Delete Host agent-4
+ sudo ovs-vsctl del-port br-central agent-4-eth0
ovs-vsctl: no port named agent-4-eth0
+ sudo ip link del agent-4-eth0
Cannot find device "agent-4-eth0"
+ sudo ip netns del agent-4
Cannot remove namespace file "/var/run/netns/agent-4": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 5 10
+ IDX=5
+ VM_NUM=10
+ NAME=agent-5
+ echo Delete agent agent-5
Delete agent agent-5
++ cat /tmp/skydive-scale/agent-5-vswitchd.pid
cat: /tmp/skydive-scale/agent-5-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-5-dbserver.pid
cat: /tmp/skydive-scale/agent-5-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm1
Cannot remove namespace file "/var/run/netns/agent-5-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm2
Cannot remove namespace file "/var/run/netns/agent-5-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm3
Cannot remove namespace file "/var/run/netns/agent-5-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm4
Cannot remove namespace file "/var/run/netns/agent-5-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm5
Cannot remove namespace file "/var/run/netns/agent-5-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm6
Cannot remove namespace file "/var/run/netns/agent-5-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm7
Cannot remove namespace file "/var/run/netns/agent-5-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm8
Cannot remove namespace file "/var/run/netns/agent-5-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm9
Cannot remove namespace file "/var/run/netns/agent-5-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm10
Cannot remove namespace file "/var/run/netns/agent-5-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-5.lock
rm: cannot remove '/tmp/skydive-scale/agent-5.lock': No such file or directory
+ delete_host agent-5
+ NAME=agent-5
+ echo Delete Host agent-5
Delete Host agent-5
+ sudo ovs-vsctl del-port br-central agent-5-eth0
ovs-vsctl: no port named agent-5-eth0
+ sudo ip link del agent-5-eth0
Cannot find device "agent-5-eth0"
+ sudo ip netns del agent-5
Cannot remove namespace file "/var/run/netns/agent-5": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 6 10
+ IDX=6
+ VM_NUM=10
+ NAME=agent-6
+ echo Delete agent agent-6
Delete agent agent-6
++ cat /tmp/skydive-scale/agent-6-vswitchd.pid
cat: /tmp/skydive-scale/agent-6-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-6-dbserver.pid
cat: /tmp/skydive-scale/agent-6-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm1
Cannot remove namespace file "/var/run/netns/agent-6-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm2
Cannot remove namespace file "/var/run/netns/agent-6-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm3
Cannot remove namespace file "/var/run/netns/agent-6-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm4
Cannot remove namespace file "/var/run/netns/agent-6-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm5
Cannot remove namespace file "/var/run/netns/agent-6-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm6
Cannot remove namespace file "/var/run/netns/agent-6-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm7
Cannot remove namespace file "/var/run/netns/agent-6-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm8
Cannot remove namespace file "/var/run/netns/agent-6-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm9
Cannot remove namespace file "/var/run/netns/agent-6-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm10
Cannot remove namespace file "/var/run/netns/agent-6-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-6.lock
rm: cannot remove '/tmp/skydive-scale/agent-6.lock': No such file or directory
+ delete_host agent-6
+ NAME=agent-6
+ echo Delete Host agent-6
Delete Host agent-6
+ sudo ovs-vsctl del-port br-central agent-6-eth0
ovs-vsctl: no port named agent-6-eth0
+ sudo ip link del agent-6-eth0
Cannot find device "agent-6-eth0"
+ sudo ip netns del agent-6
Cannot remove namespace file "/var/run/netns/agent-6": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 7 10
+ IDX=7
+ VM_NUM=10
+ NAME=agent-7
+ echo Delete agent agent-7
Delete agent agent-7
++ cat /tmp/skydive-scale/agent-7-vswitchd.pid
cat: /tmp/skydive-scale/agent-7-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-7-dbserver.pid
cat: /tmp/skydive-scale/agent-7-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm1
Cannot remove namespace file "/var/run/netns/agent-7-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm2
Cannot remove namespace file "/var/run/netns/agent-7-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm3
Cannot remove namespace file "/var/run/netns/agent-7-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm4
Cannot remove namespace file "/var/run/netns/agent-7-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm5
Cannot remove namespace file "/var/run/netns/agent-7-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm6
Cannot remove namespace file "/var/run/netns/agent-7-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm7
Cannot remove namespace file "/var/run/netns/agent-7-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm8
Cannot remove namespace file "/var/run/netns/agent-7-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm9
Cannot remove namespace file "/var/run/netns/agent-7-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm10
Cannot remove namespace file "/var/run/netns/agent-7-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-7.lock
rm: cannot remove '/tmp/skydive-scale/agent-7.lock': No such file or directory
+ delete_host agent-7
+ NAME=agent-7
+ echo Delete Host agent-7
Delete Host agent-7
+ sudo ovs-vsctl del-port br-central agent-7-eth0
ovs-vsctl: no port named agent-7-eth0
+ sudo ip link del agent-7-eth0
Cannot find device "agent-7-eth0"
+ sudo ip netns del agent-7
Cannot remove namespace file "/var/run/netns/agent-7": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 8 10
+ IDX=8
+ VM_NUM=10
+ NAME=agent-8
+ echo Delete agent agent-8
Delete agent agent-8
++ cat /tmp/skydive-scale/agent-8-vswitchd.pid
cat: /tmp/skydive-scale/agent-8-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-8-dbserver.pid
cat: /tmp/skydive-scale/agent-8-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm1
Cannot remove namespace file "/var/run/netns/agent-8-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm2
Cannot remove namespace file "/var/run/netns/agent-8-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm3
Cannot remove namespace file "/var/run/netns/agent-8-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm4
Cannot remove namespace file "/var/run/netns/agent-8-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm5
Cannot remove namespace file "/var/run/netns/agent-8-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm6
Cannot remove namespace file "/var/run/netns/agent-8-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm7
Cannot remove namespace file "/var/run/netns/agent-8-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm8
Cannot remove namespace file "/var/run/netns/agent-8-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm9
Cannot remove namespace file "/var/run/netns/agent-8-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm10
Cannot remove namespace file "/var/run/netns/agent-8-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-8.lock
rm: cannot remove '/tmp/skydive-scale/agent-8.lock': No such file or directory
+ delete_host agent-8
+ NAME=agent-8
+ echo Delete Host agent-8
Delete Host agent-8
+ sudo ovs-vsctl del-port br-central agent-8-eth0
ovs-vsctl: no port named agent-8-eth0
+ sudo ip link del agent-8-eth0
Cannot find device "agent-8-eth0"
+ sudo ip netns del agent-8
Cannot remove namespace file "/var/run/netns/agent-8": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 9 10
+ IDX=9
+ VM_NUM=10
+ NAME=agent-9
+ echo Delete agent agent-9
Delete agent agent-9
++ cat /tmp/skydive-scale/agent-9-vswitchd.pid
cat: /tmp/skydive-scale/agent-9-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-9-dbserver.pid
cat: /tmp/skydive-scale/agent-9-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm1
Cannot remove namespace file "/var/run/netns/agent-9-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm2
Cannot remove namespace file "/var/run/netns/agent-9-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm3
Cannot remove namespace file "/var/run/netns/agent-9-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm4
Cannot remove namespace file "/var/run/netns/agent-9-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm5
Cannot remove namespace file "/var/run/netns/agent-9-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm6
Cannot remove namespace file "/var/run/netns/agent-9-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm7
Cannot remove namespace file "/var/run/netns/agent-9-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm8
Cannot remove namespace file "/var/run/netns/agent-9-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm9
Cannot remove namespace file "/var/run/netns/agent-9-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm10
Cannot remove namespace file "/var/run/netns/agent-9-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-9.lock
rm: cannot remove '/tmp/skydive-scale/agent-9.lock': No such file or directory
+ delete_host agent-9
+ NAME=agent-9
+ echo Delete Host agent-9
Delete Host agent-9
+ sudo ovs-vsctl del-port br-central agent-9-eth0
ovs-vsctl: no port named agent-9-eth0
+ sudo ip link del agent-9-eth0
Cannot find device "agent-9-eth0"
+ sudo ip netns del agent-9
Cannot remove namespace file "/var/run/netns/agent-9": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 10 10
+ IDX=10
+ VM_NUM=10
+ NAME=agent-10
+ echo Delete agent agent-10
Delete agent agent-10
++ cat /tmp/skydive-scale/agent-10-vswitchd.pid
cat: /tmp/skydive-scale/agent-10-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-10-dbserver.pid
cat: /tmp/skydive-scale/agent-10-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm1
Cannot remove namespace file "/var/run/netns/agent-10-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm2
Cannot remove namespace file "/var/run/netns/agent-10-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm3
Cannot remove namespace file "/var/run/netns/agent-10-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm4
Cannot remove namespace file "/var/run/netns/agent-10-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm5
Cannot remove namespace file "/var/run/netns/agent-10-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm6
Cannot remove namespace file "/var/run/netns/agent-10-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm7
Cannot remove namespace file "/var/run/netns/agent-10-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm8
Cannot remove namespace file "/var/run/netns/agent-10-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm9
Cannot remove namespace file "/var/run/netns/agent-10-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm10
Cannot remove namespace file "/var/run/netns/agent-10-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-10.lock
rm: cannot remove '/tmp/skydive-scale/agent-10.lock': No such file or directory
+ delete_host agent-10
+ NAME=agent-10
+ echo Delete Host agent-10
Delete Host agent-10
+ sudo ovs-vsctl del-port br-central agent-10-eth0
ovs-vsctl: no port named agent-10-eth0
+ sudo ip link del agent-10-eth0
Cannot find device "agent-10-eth0"
+ sudo ip netns del agent-10
Cannot remove namespace file "/var/run/netns/agent-10": No such file or directory
+ delete_main_bridge
+ echo Delete central bridge
Delete central bridge
+ sudo ip link del br-central
Cannot find device "br-central"
+ sudo find /tmp/skydive-scale -mindepth 1 '!' -name '*.log' -a '!' -name '*.cover' -exec rm -rf '{}' ';'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to localhost port 9200: Connection refused

OrientDB console v.2.2.17 (build 2.2.x@rd9bace82ea8437117fd48114fc255e791056014b; 2017-02-16 17:20:27+0000) www.orientdb.com
Type 'help' to display all the supported commands.
Installing extensions for GREMLIN language v.2.6.0

orientdb> DROP DATABASE remote:localhost/Skydive root root plocal

Disconnecting from remote server [remote:localhost/Skydive]...
OK

Database 'remote:localhost/Skydive' deleted successfully
error: failed to get domain 'dev_dev'
error: Domain not found: no domain with matching name 'dev_dev'

error: failed to get domain 'dev_dev'
error: Domain not found: no domain with matching name 'dev_dev'

error: failed to get vol 'dev_dev.img'
error: Storage volume not found: no storage vol with matching path 'dev_dev.img'

error: failed to get domain 'vagrant_analyzer1'
error: Domain not found: no domain with matching name 'vagrant_analyzer1'

error: failed to get domain 'vagrant_analyzer1'
error: Domain not found: no domain with matching name 'vagrant_analyzer1'

error: failed to get vol 'vagrant_analyzer1.img'
error: Storage volume not found: no storage vol with matching path 'vagrant_analyzer1.img'

error: failed to get domain 'vagrant_agent1'
error: Domain not found: no domain with matching name 'vagrant_agent1'

error: failed to get domain 'vagrant_agent1'
error: Domain not found: no domain with matching name 'vagrant_agent1'

error: failed to get vol 'vagrant_agent1.img'
error: Storage volume not found: no storage vol with matching path 'vagrant_agent1.img'

error: failed to get domain 'devstack_devstack'
error: Domain not found: no domain with matching name 'devstack_devstack'

error: failed to get domain 'devstack_devstack'
error: Domain not found: no domain with matching name 'devstack_devstack'

error: failed to get vol 'devstack_devstack.img'
error: Storage volume not found: no storage vol with matching path 'devstack_devstack.img'

error: failed to get network 'vagrant0'
error: Network not found: no network with matching name 'vagrant0'

error: Failed to destroy network vagrant-libvirt
error: Requested operation is not valid: network 'vagrant-libvirt' is not active

[skydive-scale-tests] $ /bin/bash /tmp/jenkins3307513935922312314.sh

# Set Environment
echo ${PATH} | grep -q "${HOME}/bin" || {
  echo "Adding ${HOME}/bin to PATH"
  export PATH="${PATH}:${HOME}/bin"
}
Adding /var/lib/jenkins/bin to PATH

# Install Go
mkdir -p ~/bin
curl -sL -o ~/bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/master/gimme
chmod +x ~/bin/gimme

# before changing this be sure that it will not break the RHEL packaging
eval "$(gimme 1.10.3)"

unset GOOS;
unset GOARCH;
export GOROOT='/var/lib/jenkins/.gimme/versions/go1.10.3.linux.amd64';
export PATH="/var/lib/jenkins/.gimme/versions/go1.10.3.linux.amd64/bin:${PATH}";
go version >&2;
go version go1.10.3 linux/amd64

export GIMME_ENV='/var/lib/jenkins/.gimme/envs/go1.10.3.linux.amd64.env';

export GOPATH=$WORKSPACE
export PATH=$PATH:$GOPATH/bin

# speedup govendor sync command
mkdir -p $HOME/.govendor $GOPATH/.cache
rm -rf $GOPATH/.cache/govendor
ln -s $HOME/.govendor $GOPATH/.cache/govendor

# share compile cache
mkdir -p $HOME/pkg
rm -rf $GOPATH/pkg
ln -s $HOME/pkg $GOPATH/pkg
[[ "$ghprbCommentBody" = *"insanelock"* ]] && export TAGS=mutexdebug
FLOW_PROTOCOL=udp scripts/ci/run-scale.sh

dir="$(dirname "$0")"

sudo iptables -F
sudo iptables -P FORWARD ACCEPT
for i in $(find /proc/sys/net/bridge/ -type f) ; do echo 0 | sudo tee $i ; done
0
0
0
0
0
0

cd ${GOPATH}/src/github.com/skydive-project/skydive
make install
CC= GOARCH= go get github.com/kardianos/govendor
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor sync
patch -p0 < dpdk/dpdk.govendor.patch
patching file vendor/github.com/intel-go/yanff/low/low.go
rm -rf vendor/github.com/weaveworks/tcptracer-bpf/vendor/github.com/
find vendor/github.com/docker/go-connections -name "*.go" | xargs -n 1 perl -i -pe 's|github.com/Sirupsen|github.com/sirupsen|g'
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/gopkg.in/src-d/proteus.v1/cli/proteus; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install gopkg.in/src-d/proteus.v1/cli/proteus); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; proteus proto -f ${GOPATH}/src -p github.com/skydive-project/skydive/flow/layers
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
sed -e 's/^package .*;/package layers;/' -i flow/layers/generated.proto
sed -e 's/^option go_package = "layers"/option go_package = "github.com\/skydive-project\/skydive\/flow\/layers"/' -i flow/layers/generated.proto
sed -e 's/^message Layer/message /' -i flow/layers/generated.proto
sed -e 's/option (gogoproto.typedecl) = false;//' -i flow/layers/generated.proto
sed 's/\((gogoproto\.customname) = "\([^\"]*\)"\)/\1, (gogoproto.jsontag) = "\2,omitempty"/' -i flow/layers/generated.proto
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/gogo/protobuf/protoc-gen-gogofaster; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/gogo/protobuf/protoc-gen-gogofaster); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor;
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/golang/protobuf/protoc-gen-go; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/golang/protobuf/protoc-gen-go); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; protoc -Ivendor -I. --plugin=/var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool/protoc-gen-gogofaster --gogofaster_out $GOPATH/src flow/layers/generated.proto
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/gogo/protobuf/protoc-gen-gogofaster; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/gogo/protobuf/protoc-gen-gogofaster); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor;
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/golang/protobuf/protoc-gen-go; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/golang/protobuf/protoc-gen-go); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; protoc -Ivendor -I. --plugin=/var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool/protoc-gen-gogofaster --gogofaster_out $GOPATH/src flow/flow.proto
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
# always export flow.ParentUUID as we need to store this information to know
# if it's a Outer or Inner packet.
sed -e 's/ParentUUID\(.*\),omitempty\(.*\)/ParentUUID\1\2/' \
	-e 's/Protocol\(.*\),omitempty\(.*\)/Protocol\1\2/' \
	-e 's/ICMPType\(.*\),omitempty\(.*\)/ICMPType\1\2/' \
	-e 's/int64\(.*\),omitempty\(.*\)/int64\1\2/' \
	-i flow/flow.pb.go
# add omitempty to RTT as it is not always filled
sed -e 's/json:"RTT"/json:"RTT,omitempty"/' -i flow/flow.pb.go
# do not export LastRawPackets used internally
sed -e 's/json:"LastRawPackets,omitempty"/json:"-"/g' -i flow/flow.pb.go
# add flowState to flow generated struct
sed -e 's/type Flow struct {/type Flow struct { XXX_state flowState `json:"-"`/' -i flow/flow.pb.go
# to fix generated layers import
sed -e 's/layers "flow\/layers"/layers "github.com\/skydive-project\/skydive\/flow\/layers"/' -i flow/flow.pb.go
gofmt -s -w flow/flow.pb.go
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/gogo/protobuf/protoc-gen-gogofaster; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/gogo/protobuf/protoc-gen-gogofaster); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor;
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/golang/protobuf/protoc-gen-go; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/golang/protobuf/protoc-gen-go); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; protoc -Ivendor -I. --plugin=/var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool/protoc-gen-gogofaster --gogofaster_out $GOPATH/src filters/filters.proto
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/gogo/protobuf/protoc-gen-gogofaster; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/gogo/protobuf/protoc-gen-gogofaster); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor;
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/golang/protobuf/protoc-gen-go; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/golang/protobuf/protoc-gen-go); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; protoc -Ivendor -I. --plugin=/var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool/protoc-gen-gogofaster --gogofaster_out $GOPATH/src websocket/structmessage.proto
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
sed -e 's/type StructMessage struct {/type StructMessage struct { XXX_state structMessageState `json:"-"`/' -i websocket/structmessage.pb.go
gofmt -s -w websocket/structmessage.pb.go
cd js && npm install
npm notice created a lockfile as package-lock.json. You should commit this file.
added 148 packages from 122 contributors and audited 930 packages in 48.087s
found 0 vulnerabilities

cd js && PATH=`npm bin`:$PATH make all
make[1]: Entering directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/js'
tsc --module commonjs --target ES5 browser.ts
browserify browser.js -o ../statics/js/bundle.js
make[1]: Leaving directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/js'
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/jteeuwen/go-bindata/go-bindata; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/jteeuwen/go-bindata/go-bindata); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; go-bindata  -nometadata -o statics/bindata.go -pkg=statics -ignore=bindata.go js/*.js rbac/policy.csv statics/index.html statics/css/* statics/css/themes/*/* statics/fonts/* statics/img/* statics/js/* statics/schemas/* statics/workflows/*.yaml 
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
gofmt -w -s statics/bindata.go
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson -all flow/flow.pb.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson flow/storage/elasticsearch/elasticsearch.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson flow/storage/orientdb/orientdb.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson graffiti/graph/elasticsearch.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson sflow/sflow.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson topology/metrics.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson topology/probes/netlink/route.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson topology/probes/netlink/neighbor.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson topology/probes/ovsdb/ovsdb.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson -build_tags linux topology/probes/netlink/netlink.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson -build_tags linux topology/probes/socketinfo/connection.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/mailru/easyjson/easyjson; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/mailru/easyjson/easyjson); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; easyjson -build_tags opencontrail topology/probes/opencontrail/routing_table.go
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/git.fd.io/govpp.git/cmd/binapi-generator; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install git.fd.io/govpp.git/cmd/binapi-generator); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor;
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor generate -tags " opencontrail lxd libvirt" \
	github.com/skydive-project/skydive/topology/probes/vpp
CGO_CFLAGS_ALLOW='.*' CGO_LDFLAGS_ALLOW='.*' /var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor install \
	-ldflags=" -B 0x0ae29e90f3a3ae751def24c3f33baaa8bb2f5f18 -X github.com/skydive-project/skydive/version.Version=0.22.0-5bebd28d8830" \
	 -tags=" opencontrail lxd libvirt" -v \
	github.com/skydive-project/skydive
github.com/skydive-project/skydive/api/server
github.com/skydive-project/skydive/alert
github.com/skydive-project/skydive/topology/enhancers
github.com/skydive-project/skydive/flow/ondemand/client
github.com/skydive-project/skydive/packetinjector
github.com/skydive-project/skydive/graffiti/pod
github.com/skydive-project/skydive/analyzer
github.com/skydive-project/skydive/agent

export SKYDIVE_ANALYZERS=localhost:8082
export ELASTICSEARCH=localhost:9200
export TLS=true
export SKYDIVE=${GOPATH}/bin/skydive
export FLOW_PROTOCOL=${FLOW_PROTOCOL:-websocket}
export SKYDIVE_LOGGING_LEVEL=DEBUG

make test.functionals WITH_SCALE=true TAGS=${TAGS} VERBOSE=true TIMEOUT=10m TEST_PATTERN=Scale
CC= GOARCH= go get github.com/kardianos/govendor
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor sync
patch -p0 < dpdk/dpdk.govendor.patch
patching file vendor/github.com/intel-go/yanff/low/low.go
rm -rf vendor/github.com/weaveworks/tcptracer-bpf/vendor/github.com/
find vendor/github.com/docker/go-connections -name "*.go" | xargs -n 1 perl -i -pe 's|github.com/Sirupsen|github.com/sirupsen|g'
cd js && npm install
audited 930 packages in 9.277s
found 0 vulnerabilities

cd js && PATH=`npm bin`:$PATH make all
make[1]: Entering directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/js'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/js'
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/jteeuwen/go-bindata/go-bindata; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/jteeuwen/go-bindata/go-bindata); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; go-bindata  -nometadata -o statics/bindata.go -pkg=statics -ignore=bindata.go js/*.js rbac/policy.csv statics/index.html statics/css/* statics/css/themes/*/* statics/fonts/* statics/img/* statics/js/* statics/schemas/* statics/workflows/*.yaml 
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
gofmt -w -s statics/bindata.go
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/git.fd.io/govpp.git/cmd/binapi-generator; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install git.fd.io/govpp.git/cmd/binapi-generator); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor;
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor generate -tags " scale opencontrail lxd libvirt" \
	github.com/skydive-project/skydive/topology/probes/vpp
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor test -tags " scale opencontrail lxd libvirt test"  -v -timeout 10m -c -o tests/functionals ./tests/
for functest in TestScaleHA ; do \
	make ARGS="-test.run $functest$\$ " test.functionals.run EXTRA_ARGS="-analyzer.topology.probes= -agent.topology.probes=opencontrail"; \
done
make[1]: Entering directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive'
cd tests && sudo -E ./functionals -test.v -test.timeout 10m -test.run TestScaleHA$  -analyzer.topology.probes= -agent.topology.probes=opencontrail
Config: ---
http:
  ws:
    pong_timeout: 10

analyzers:
  - 127.0.0.1:64500

analyzer:
  listen: 0.0.0.0:64500
  flow:
    backend: 
  analyzer_username: admin
  analyzer_password: password
  topology:
    backend: memory
    probes: 

  startup:
    capture_gremlin: "g.V().Has('Name','startup-vm2')"

agent:
  listen: 0.0.0.0:64499
  topology:
    probes:
    - ovsdb
    - docker
    - lxd
    - lldp
    - runc
    - socketinfo
    - libvirt
    - opencontrail

    netlink:
      metrics_update: 5
    lldp:
      interfaces:
      - lldp0

  metadata:
    mydict:
      value: 123
    myarrays:
      integers:
      - 1
      - 2
      - 3
      bools:
      - true
      - true
      strings:
      - dog
      - cat
      - frog

flow:
  expire: 600
  update: 10

ovs:
  oflow:
    enable: true
    native: false
    address:
      br-test: tcp://127.0.0.1:16633

storage:
  orientdb:
    addr: http://127.0.0.1:2480
    database: Skydive
    username: root
    password: 

logging:
  level: DEBUG

etcd:
  data_dir: /tmp/skydive-etcd
  embedded: true
  servers:
    - http://localhost:12379

=== RUN   TestScaleHA
2019-05-15T16:03:29.675Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh start 2 2 2]
2019-05-15T16:03:36.470Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' start == start ']'
+ '[' -z 2 ']'
+ start 2 2 2
+ ANALYZER_NUM=2
+ AGENT_NUM=2
+ VM_NUM=2
+ '[' '!' -e /tmp/skydive-scale/screen.lock ']'
+ sudo screen -dmS skydive-stress
+ sudo screen -ls
There is a screen on:
	13504.skydive-stress	(Detached)
1 Socket in /run/screen/S-root.
+ touch /tmp/skydive-scale/screen.lock
+ '[' true = true ']'
+ generate_ssl_conf 2 2
+ ANALYZER_NUM=2
+ AGENT_NUM=2
+ cat
++ seq 2
+ for I in $( seq $ANALYZER_NUM )
+ IDX=4
+ echo IP.4 = 192.168.50.1
+ for I in $( seq $ANALYZER_NUM )
+ IDX=5
+ echo IP.5 = 192.168.50.2
+ TOTAL_AGENT=7
++ seq 7
+ for I in $( seq $TOTAL_AGENT )
+ IP=3
+ IDX=5
+ echo IP.5 = 192.168.50.3
+ for I in $( seq $TOTAL_AGENT )
+ IP=4
+ IDX=6
+ echo IP.6 = 192.168.50.4
+ for I in $( seq $TOTAL_AGENT )
+ IP=5
+ IDX=7
+ echo IP.7 = 192.168.50.5
+ for I in $( seq $TOTAL_AGENT )
+ IP=6
+ IDX=8
+ echo IP.8 = 192.168.50.6
+ for I in $( seq $TOTAL_AGENT )
+ IP=7
+ IDX=9
+ echo IP.9 = 192.168.50.7
+ for I in $( seq $TOTAL_AGENT )
+ IP=8
+ IDX=10
+ echo IP.10 = 192.168.50.8
+ for I in $( seq $TOTAL_AGENT )
+ IP=9
+ IDX=11
+ echo IP.11 = 192.168.50.9
+ generate_tls_crt analyzer
+ NAME=analyzer
+ '[' -e /tmp/skydive-scale/analyzer.crt ']'
+ '[' '!' -e /tmp/skydive-scale/rootCA.crt ']'
+ sudo openssl genrsa -out /tmp/skydive-scale/rootCA.key 4096
Generating RSA private key, 4096 bit long modulus (2 primes)
...............................................................................................++++
......++++
e is 65537 (0x010001)
+ sudo chmod 400 /tmp/skydive-scale/rootCA.key
+ sudo openssl req -x509 -new -nodes -key /tmp/skydive-scale/rootCA.key -days 365 -out /tmp/skydive-scale/rootCA.crt
+ yes ''
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:State or Province Name (full name) []:Locality Name (eg, city) [Default City]:Organization Name (eg, company) [Default Company Ltd]:Organizational Unit Name (eg, section) []:Common Name (eg, your name or your server's hostname) []:Email Address []:+ sudo chmod 444 /tmp/skydive-scale/rootCA.crt
+ sudo openssl genrsa -out /tmp/skydive-scale/analyzer.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
................+++++
...............+++++
e is 65537 (0x010001)
+ sudo chmod 400 /tmp/skydive-scale/analyzer.key
+ sudo openssl req -new -key /tmp/skydive-scale/analyzer.key -out /tmp/skydive-scale/analyzer.csr -subj /CN=analyzer -config /tmp/skydive-scale/skydive-ssl.cnf
+ yes ''
+ sudo openssl x509 -req -days 365 -in /tmp/skydive-scale/analyzer.csr -CA /tmp/skydive-scale/rootCA.crt -CAkey /tmp/skydive-scale/rootCA.key -CAcreateserial -out /tmp/skydive-scale/analyzer.crt -extfile /tmp/skydive-scale/skydive-ssl.cnf -extensions v3_req
Signature ok
subject=CN = analyzer
Getting CA Private Key
+ sudo chmod 444 /tmp/skydive-scale/analyzer.crt
+ generate_tls_crt agent
+ NAME=agent
+ '[' -e /tmp/skydive-scale/agent.crt ']'
+ '[' '!' -e /tmp/skydive-scale/rootCA.crt ']'
+ sudo openssl genrsa -out /tmp/skydive-scale/agent.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
..+++++
................................................................................+++++
e is 65537 (0x010001)
+ sudo chmod 400 /tmp/skydive-scale/agent.key
+ yes ''
+ sudo openssl req -new -key /tmp/skydive-scale/agent.key -out /tmp/skydive-scale/agent.csr -subj /CN=agent -config /tmp/skydive-scale/skydive-ssl.cnf
+ sudo openssl x509 -req -days 365 -in /tmp/skydive-scale/agent.csr -CA /tmp/skydive-scale/rootCA.crt -CAkey /tmp/skydive-scale/rootCA.key -CAcreateserial -out /tmp/skydive-scale/agent.crt -extfile /tmp/skydive-scale/skydive-ssl.cnf -extensions v3_req
Signature ok
subject=CN = agent
Getting CA Private Key
+ sudo chmod 444 /tmp/skydive-scale/agent.crt
+ create_main_bridge
+ echo Create central bridge
Create central bridge
+ sudo brctl addbr br-central
+ sudo brctl stp br-central off
+ sudo ip l set br-central up
+ sudo ip a add 192.168.50.254/24 dev br-central
+ start_inotify
+ '[' -f /tmp/skydive-scale/inotify.lock ']'
+ echo Start inotify netns watcher
Start inotify netns watcher
+ sudo mkdir -p /var/run/netns
+ cat
+ sudo -E screen -S skydive-stress -X screen -t skydive-inotify 0 bash -x /tmp/skydive-scale/inotify.sh
+ touch /tmp/skydive-scale/inotify.lock
+ WINDOW=1
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 1 -gt 1 ']'
+ '[' '!' -f /tmp/skydive-scale/analyzer-1.lock ']'
+ create_analyzer 1 2 2
+ IDX=1
+ ANALYZER_NUM=2
+ AGENT_NUM=2
+ NAME=analyzer-1
+ echo Create analyzer analyzer-1
Create analyzer analyzer-1
+ mkdir -p /tmp/skydive-scale/analyzer-1-etcd
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - localhost:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - localhost:8084'
+ STORAGE=
+ GRAPH=memory
+ '[' xlocalhost:9200 '!=' x ']'
+ STORAGE=elasticsearch
+ GRAPH=elasticsearch
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ cat
+ TOTAL_AGENT=7
++ seq 7
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_1'
+ echo '      - TOR_PORT_1 --> *[Name=agent-1]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_2'
+ echo '      - TOR_PORT_2 --> *[Name=agent-2]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_3'
+ echo '      - TOR_PORT_3 --> *[Name=agent-3]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_4'
+ echo '      - TOR_PORT_4 --> *[Name=agent-4]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_5'
+ echo '      - TOR_PORT_5 --> *[Name=agent-5]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_6'
+ echo '      - TOR_PORT_6 --> *[Name=agent-6]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_7'
+ echo '      - TOR_PORT_7 --> *[Name=agent-7]/eth0'
+ cat
++ seq 7
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-1: agent-password-1'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-2: agent-password-2'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-3: agent-password-3'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-4: agent-password-4'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-5: agent-password-5'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-6: agent-password-6'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-7: agent-password-7'
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-1: analyzer-password-1'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-2: analyzer-password-2'
+ COVERFILE=/tmp/skydive-scale/analyzer-1.cover
+ PIDFILE=/tmp/skydive-scale/analyzer-1.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-analyzer 1 sh -c 'export COVERFILE=/tmp/skydive-scale/analyzer-1.cover ; export PIDFILE=/tmp/skydive-scale/analyzer-1.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive analyzer -c /tmp/skydive-scale/analyzer-1.yml 2>&1 | tee /tmp/skydive-scale/analyzer-1.log'
+ touch /tmp/skydive-scale/analyzer-1.lock
+ CURR_ANALYZER_PORT=8084
+ WINDOW=2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 2 -gt 1 ']'
+ ETCD_EMBEDDED=false
+ '[' '!' -f /tmp/skydive-scale/analyzer-2.lock ']'
+ create_analyzer 2 2 2
+ IDX=2
+ ANALYZER_NUM=2
+ AGENT_NUM=2
+ NAME=analyzer-2
+ echo Create analyzer analyzer-2
Create analyzer analyzer-2
+ mkdir -p /tmp/skydive-scale/analyzer-2-etcd
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - localhost:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - localhost:8084'
+ STORAGE=
+ GRAPH=memory
+ '[' xlocalhost:9200 '!=' x ']'
+ STORAGE=elasticsearch
+ GRAPH=elasticsearch
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ cat
+ TOTAL_AGENT=7
++ seq 7
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_1'
+ echo '      - TOR_PORT_1 --> *[Name=agent-1]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_2'
+ echo '      - TOR_PORT_2 --> *[Name=agent-2]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_3'
+ echo '      - TOR_PORT_3 --> *[Name=agent-3]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_4'
+ echo '      - TOR_PORT_4 --> *[Name=agent-4]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_5'
+ echo '      - TOR_PORT_5 --> *[Name=agent-5]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_6'
+ echo '      - TOR_PORT_6 --> *[Name=agent-6]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_7'
+ echo '      - TOR_PORT_7 --> *[Name=agent-7]/eth0'
+ cat
++ seq 7
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-1: agent-password-1'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-2: agent-password-2'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-3: agent-password-3'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-4: agent-password-4'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-5: agent-password-5'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-6: agent-password-6'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-7: agent-password-7'
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-1: analyzer-password-1'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-2: analyzer-password-2'
+ COVERFILE=/tmp/skydive-scale/analyzer-2.cover
+ PIDFILE=/tmp/skydive-scale/analyzer-2.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-analyzer 2 sh -c 'export COVERFILE=/tmp/skydive-scale/analyzer-2.cover ; export PIDFILE=/tmp/skydive-scale/analyzer-2.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive analyzer -c /tmp/skydive-scale/analyzer-2.yml 2>&1 | tee /tmp/skydive-scale/analyzer-2.log'
+ touch /tmp/skydive-scale/analyzer-2.lock
+ CURR_ANALYZER_PORT=8086
+ WINDOW=3
+ AGENTS=
++ seq 2
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-1.lock ']'
+ create_host agent-1 192.168.50.1
+ NAME=agent-1
+ ADDR=192.168.50.1
+ echo Create host agent-1
Create host agent-1
+ sudo ip netns add agent-1
+ sudo ip netns exec agent-1 ip link set lo up
+ sudo ip link add agent-1-eth0 type veth peer name eth0 netns agent-1
+ sudo ip link set agent-1-eth0 up
+ sudo ip netns exec agent-1 ip link set eth0 up
+ sudo ip netns exec agent-1 ip address add 192.168.50.1/24 dev eth0
+ sudo ip netns exec agent-1 ip route add 0.0.0.0/0 via 192.168.50.254
+ sudo ip netns exec agent-1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-1-eth0
+ create_agent 1 192.168.50.1 2 2
+ IDX=1
+ ADDR=192.168.50.1
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-1
+ echo Create agent agent-1
Create agent agent-1
+ sudo ip netns exec agent-1 ovsdb-tool create /tmp/skydive-scale/agent-1.db
+ sudo ip netns exec agent-1 ovsdb-server /tmp/skydive-scale/agent-1.db --remote=punix:/tmp/skydive-scale/agent-1.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-1-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-1-dbserver.pid --detach
2019-05-15T16:03:32Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-1-vswitchd.log
+ sudo ip netns exec agent-1 ovs-vswitchd unix:/tmp/skydive-scale/agent-1.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-1-vswitchd.log --pidfile=/tmp/skydive-scale/agent-1-vswitchd.pid --detach --monitor
+ mkdir -p /tmp/skydive-scale/agent-1-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-1.cover
+ PIDFILE=/tmp/skydive-scale/agent-1.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 3 ip netns exec agent-1 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-1.cover ; export PIDFILE=/tmp/skydive-scale/agent-1.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-1.yml 2>&1 | tee /tmp/skydive-scale/agent-1.log'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-br agent-1
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock set bridge agent-1 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-1-vm1
Create VM agent-1-vm1
+ sudo ip netns add agent-1-vm1
+ sudo ip netns exec agent-1-vm1 ip l set lo up
+ sudo ip netns exec agent-1 ip link add vm1-eth0 type veth peer name eth0 netns agent-1-vm1
+ sudo ip netns exec agent-1 ip link set vm1-eth0 up
+ sudo ip netns exec agent-1-vm1 ip link set eth0 up
+ sudo ip netns exec agent-1-vm1 ip address add 10.0.1.1/16 dev eth0
+ sudo ip netns exec agent-1-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 vm1-eth0
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-1-vm2
Create VM agent-1-vm2
+ sudo ip netns add agent-1-vm2
+ sudo ip netns exec agent-1-vm2 ip l set lo up
+ sudo ip netns exec agent-1 ip link add vm2-eth0 type veth peer name eth0 netns agent-1-vm2
+ sudo ip netns exec agent-1 ip link set vm2-eth0 up
+ sudo ip netns exec agent-1-vm2 ip link set eth0 up
+ sudo ip netns exec agent-1-vm2 ip address add 10.0.1.2/16 dev eth0
+ sudo ip netns exec agent-1-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 vm2-eth0
+ touch /tmp/skydive-scale/agent-1.lock
+ '[' -z '' ']'
+ AGENTS=192.168.50.1
+ WINDOW=4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-2.lock ']'
+ create_host agent-2 192.168.50.2
+ NAME=agent-2
+ ADDR=192.168.50.2
+ echo Create host agent-2
Create host agent-2
+ sudo ip netns add agent-2
+ sudo ip netns exec agent-2 ip link set lo up
+ sudo ip link add agent-2-eth0 type veth peer name eth0 netns agent-2
+ sudo ip link set agent-2-eth0 up
+ sudo ip netns exec agent-2 ip link set eth0 up
+ sudo ip netns exec agent-2 ip address add 192.168.50.2/24 dev eth0
+ sudo ip netns exec agent-2 ip route add 0.0.0.0/0 via 192.168.50.254
+ sudo ip netns exec agent-2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-2-eth0
+ create_agent 2 192.168.50.2 2 2
+ IDX=2
+ ADDR=192.168.50.2
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-2
+ echo Create agent agent-2
Create agent agent-2
+ sudo ip netns exec agent-2 ovsdb-tool create /tmp/skydive-scale/agent-2.db
+ sudo ip netns exec agent-2 ovsdb-server /tmp/skydive-scale/agent-2.db --remote=punix:/tmp/skydive-scale/agent-2.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-2-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-2-dbserver.pid --detach
2019-05-15T16:03:34Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-2-vswitchd.log
+ sudo ip netns exec agent-2 ovs-vswitchd unix:/tmp/skydive-scale/agent-2.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-2-vswitchd.log --pidfile=/tmp/skydive-scale/agent-2-vswitchd.pid --detach --monitor
+ mkdir -p /tmp/skydive-scale/agent-2-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-2.cover
+ PIDFILE=/tmp/skydive-scale/agent-2.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 4 ip netns exec agent-2 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-2.cover ; export PIDFILE=/tmp/skydive-scale/agent-2.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-2.yml 2>&1 | tee /tmp/skydive-scale/agent-2.log'
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-br agent-2
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock set bridge agent-2 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-2-vm1
Create VM agent-2-vm1
+ sudo ip netns add agent-2-vm1
+ sudo ip netns exec agent-2-vm1 ip l set lo up
+ sudo ip netns exec agent-2 ip link add vm1-eth0 type veth peer name eth0 netns agent-2-vm1
+ sudo ip netns exec agent-2 ip link set vm1-eth0 up
+ sudo ip netns exec agent-2-vm1 ip link set eth0 up
+ sudo ip netns exec agent-2-vm1 ip address add 10.0.2.1/16 dev eth0
+ sudo ip netns exec agent-2-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 vm1-eth0
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-2-vm2
Create VM agent-2-vm2
+ sudo ip netns add agent-2-vm2
+ sudo ip netns exec agent-2-vm2 ip l set lo up
+ sudo ip netns exec agent-2 ip link add vm2-eth0 type veth peer name eth0 netns agent-2-vm2
+ sudo ip netns exec agent-2 ip link set vm2-eth0 up
+ sudo ip netns exec agent-2-vm2 ip link set eth0 up
+ sudo ip netns exec agent-2-vm2 ip address add 10.0.2.2/16 dev eth0
+ sudo ip netns exec agent-2-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 vm2-eth0
+ touch /tmp/skydive-scale/agent-2.lock
+ '[' -z 192.168.50.1 ']'
+ AGENTS=192.168.50.1,192.168.50.2
+ WINDOW=5
++ seq 2
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-1 192.168.50.1 192.168.50.1,192.168.50.2
+ NAME=agent-1
+ ADDR=192.168.50.1
+ AGENTS=192.168.50.1,192.168.50.2
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.1 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.2 ']'
+ echo '192.168.50.1 -> 192.168.50.2'
192.168.50.1 -> 192.168.50.2
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-2 192.168.50.2 192.168.50.1,192.168.50.2
+ NAME=agent-2
+ ADDR=192.168.50.2
+ AGENTS=192.168.50.1,192.168.50.2
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.1 ']'
+ echo '192.168.50.2 -> 192.168.50.1'
192.168.50.2 -> 192.168.50.1
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.2 ']'
+ echo Go to : http://192.168.50.254:8082
Go to : http://192.168.50.254:8082

2019-05-15T16:03:36.470Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [sleep 30]
2019-05-15T16:04:06.476Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned 
2019-05-15T16:04:06.773Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:06.932Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=1.86 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.857/1.857/1.857/0.000 ms

2019-05-15T16:04:07.933Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:08.062Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.177 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.177/0.177/0.177/0.000 ms

2019-05-15T16:04:09.062Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:09.267Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.252 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.252/0.252/0.252/0.000 ms

2019-05-15T16:04:10.267Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:10.391Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.161 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.161/0.161/0.161/0.000 ms

2019-05-15T16:04:11.391Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:11.500Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.133 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.133/0.133/0.133/0.000 ms

2019-05-15T16:04:12.500Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:12.632Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.223 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.223/0.223/0.223/0.000 ms

2019-05-15T16:04:13.632Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:13.761Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.187 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.187/0.187/0.187/0.000 ms

2019-05-15T16:04:14.761Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:14.893Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.177 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.177/0.177/0.177/0.000 ms

2019-05-15T16:04:15.893Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:16.035Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.302 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.302/0.302/0.302/0.000 ms

2019-05-15T16:04:17.036Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:17.163Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.198 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.198/0.198/0.198/0.000 ms

2019-05-15T16:04:18.163Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:18.289Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.123 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.123/0.123/0.123/0.000 ms

2019-05-15T16:04:19.290Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:19.436Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.170 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.170/0.170/0.170/0.000 ms

2019-05-15T16:04:20.437Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:20.582Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.124 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.124/0.124/0.124/0.000 ms

2019-05-15T16:04:21.582Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:21.703Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.110 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.110/0.110/0.110/0.000 ms

2019-05-15T16:04:22.704Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:22.833Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.120 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.120/0.120/0.120/0.000 ms

2019-05-15T16:04:23.833Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:24.021Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ awk '{print $2}'
++ grep 'inet '
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.131 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.131/0.131/0.131/0.000 ms

2019-05-15T16:04:25.022Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:25.183Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.287 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.287/0.287/0.287/0.000 ms

2019-05-15T16:04:26.183Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:26.321Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.144 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.144/0.144/0.144/0.000 ms

2019-05-15T16:04:27.322Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:27.448Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.113 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.113/0.113/0.113/0.000 ms

2019-05-15T16:04:28.449Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:28.578Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.122 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.122/0.122/0.122/0.000 ms

2019-05-15T16:04:29.579Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:29.706Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.151 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.151/0.151/0.151/0.000 ms

2019-05-15T16:04:30.707Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:30.861Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.151 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.151/0.151/0.151/0.000 ms

2019-05-15T16:04:31.862Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:31.985Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.157 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.157/0.157/0.157/0.000 ms

2019-05-15T16:04:32.985Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:33.110Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.198 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.198/0.198/0.198/0.000 ms

2019-05-15T16:04:34.110Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:34.227Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.122 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.122/0.122/0.122/0.000 ms

2019-05-15T16:04:35.227Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:35.363Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.152 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.152/0.152/0.152/0.000 ms

2019-05-15T16:04:36.363Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:36.476Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.134 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.134/0.134/0.134/0.000 ms

2019-05-15T16:04:37.477Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:37.596Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.173 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.173/0.173/0.173/0.000 ms

2019-05-15T16:04:38.596Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:38.723Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.239 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.239/0.239/0.239/0.000 ms

2019-05-15T16:04:39.724Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:04:39.841Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.174 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.174/0.174/0.174/0.000 ms

2019-05-15T16:04:50.654Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh start 2 4 2]
2019-05-15T16:04:57.211Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' start == start ']'
+ '[' -z 2 ']'
+ start 2 4 2
+ ANALYZER_NUM=2
+ AGENT_NUM=4
+ VM_NUM=2
+ '[' '!' -e /tmp/skydive-scale/screen.lock ']'
+ '[' true = true ']'
+ generate_ssl_conf 2 4
+ ANALYZER_NUM=2
+ AGENT_NUM=4
+ cat
++ seq 2
+ for I in $( seq $ANALYZER_NUM )
+ IDX=4
+ echo IP.4 = 192.168.50.1
+ for I in $( seq $ANALYZER_NUM )
+ IDX=5
+ echo IP.5 = 192.168.50.2
+ TOTAL_AGENT=9
++ seq 9
+ for I in $( seq $TOTAL_AGENT )
+ IP=3
+ IDX=5
+ echo IP.5 = 192.168.50.3
+ for I in $( seq $TOTAL_AGENT )
+ IP=4
+ IDX=6
+ echo IP.6 = 192.168.50.4
+ for I in $( seq $TOTAL_AGENT )
+ IP=5
+ IDX=7
+ echo IP.7 = 192.168.50.5
+ for I in $( seq $TOTAL_AGENT )
+ IP=6
+ IDX=8
+ echo IP.8 = 192.168.50.6
+ for I in $( seq $TOTAL_AGENT )
+ IP=7
+ IDX=9
+ echo IP.9 = 192.168.50.7
+ for I in $( seq $TOTAL_AGENT )
+ IP=8
+ IDX=10
+ echo IP.10 = 192.168.50.8
+ for I in $( seq $TOTAL_AGENT )
+ IP=9
+ IDX=11
+ echo IP.11 = 192.168.50.9
+ for I in $( seq $TOTAL_AGENT )
+ IP=10
+ IDX=12
+ echo IP.12 = 192.168.50.10
+ for I in $( seq $TOTAL_AGENT )
+ IP=11
+ IDX=13
+ echo IP.13 = 192.168.50.11
+ generate_tls_crt analyzer
+ NAME=analyzer
+ '[' -e /tmp/skydive-scale/analyzer.crt ']'
+ return
+ generate_tls_crt agent
+ NAME=agent
+ '[' -e /tmp/skydive-scale/agent.crt ']'
+ return
+ create_main_bridge
+ echo Create central bridge
Create central bridge
+ sudo brctl addbr br-central
device br-central already exists; can't create bridge with the same name
+ sudo brctl stp br-central off
+ sudo ip l set br-central up
+ sudo ip a add 192.168.50.254/24 dev br-central
RTNETLINK answers: File exists
+ start_inotify
+ '[' -f /tmp/skydive-scale/inotify.lock ']'
+ return
+ WINDOW=1
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 1 -gt 1 ']'
+ '[' '!' -f /tmp/skydive-scale/analyzer-1.lock ']'
+ CURR_ANALYZER_PORT=8084
+ WINDOW=2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 2 -gt 1 ']'
+ ETCD_EMBEDDED=false
+ '[' '!' -f /tmp/skydive-scale/analyzer-2.lock ']'
+ CURR_ANALYZER_PORT=8086
+ WINDOW=3
+ AGENTS=
++ seq 4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-1.lock ']'
+ '[' -z '' ']'
+ AGENTS=192.168.50.1
+ WINDOW=4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-2.lock ']'
+ '[' -z 192.168.50.1 ']'
+ AGENTS=192.168.50.1,192.168.50.2
+ WINDOW=5
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-3.lock ']'
+ create_host agent-3 192.168.50.3
+ NAME=agent-3
+ ADDR=192.168.50.3
+ echo Create host agent-3
Create host agent-3
+ sudo ip netns add agent-3
+ sudo ip netns exec agent-3 ip link set lo up
+ sudo ip link add agent-3-eth0 type veth peer name eth0 netns agent-3
+ sudo ip link set agent-3-eth0 up
+ sudo ip netns exec agent-3 ip link set eth0 up
+ sudo ip netns exec agent-3 ip address add 192.168.50.3/24 dev eth0
+ sudo ip netns exec agent-3 ip route add 0.0.0.0/0 via 192.168.50.254
+ sudo ip netns exec agent-3 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-3-eth0
+ create_agent 3 192.168.50.3 2 2
+ IDX=3
+ ADDR=192.168.50.3
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-3
+ echo Create agent agent-3
Create agent agent-3
+ sudo ip netns exec agent-3 ovsdb-tool create /tmp/skydive-scale/agent-3.db
+ sudo ip netns exec agent-3 ovsdb-server /tmp/skydive-scale/agent-3.db --remote=punix:/tmp/skydive-scale/agent-3.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-3-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-3-dbserver.pid --detach
2019-05-15T16:04:51Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-3-vswitchd.log
+ sudo ip netns exec agent-3 ovs-vswitchd unix:/tmp/skydive-scale/agent-3.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-3-vswitchd.log --pidfile=/tmp/skydive-scale/agent-3-vswitchd.pid --detach --monitor
+ mkdir -p /tmp/skydive-scale/agent-3-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-3.cover
+ PIDFILE=/tmp/skydive-scale/agent-3.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 5 ip netns exec agent-3 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-3.cover ; export PIDFILE=/tmp/skydive-scale/agent-3.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-3.yml 2>&1 | tee /tmp/skydive-scale/agent-3.log'
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-br agent-3
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock set bridge agent-3 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-3-vm1
Create VM agent-3-vm1
+ sudo ip netns add agent-3-vm1
+ sudo ip netns exec agent-3-vm1 ip l set lo up
+ sudo ip netns exec agent-3 ip link add vm1-eth0 type veth peer name eth0 netns agent-3-vm1
+ sudo ip netns exec agent-3 ip link set vm1-eth0 up
+ sudo ip netns exec agent-3-vm1 ip link set eth0 up
+ sudo ip netns exec agent-3-vm1 ip address add 10.0.3.1/16 dev eth0
+ sudo ip netns exec agent-3-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 vm1-eth0
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-3-vm2
Create VM agent-3-vm2
+ sudo ip netns add agent-3-vm2
+ sudo ip netns exec agent-3-vm2 ip l set lo up
+ sudo ip netns exec agent-3 ip link add vm2-eth0 type veth peer name eth0 netns agent-3-vm2
+ sudo ip netns exec agent-3 ip link set vm2-eth0 up
+ sudo ip netns exec agent-3-vm2 ip link set eth0 up
+ sudo ip netns exec agent-3-vm2 ip address add 10.0.3.2/16 dev eth0
+ sudo ip netns exec agent-3-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 vm2-eth0
+ touch /tmp/skydive-scale/agent-3.lock
+ '[' -z 192.168.50.1,192.168.50.2 ']'
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ WINDOW=6
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-4.lock ']'
+ create_host agent-4 192.168.50.4
+ NAME=agent-4
+ ADDR=192.168.50.4
+ echo Create host agent-4
Create host agent-4
+ sudo ip netns add agent-4
+ sudo ip netns exec agent-4 ip link set lo up
+ sudo ip link add agent-4-eth0 type veth peer name eth0 netns agent-4
+ sudo ip link set agent-4-eth0 up
+ sudo ip netns exec agent-4 ip link set eth0 up
+ sudo ip netns exec agent-4 ip address add 192.168.50.4/24 dev eth0
+ sudo ip netns exec agent-4 ip route add 0.0.0.0/0 via 192.168.50.254
+ sudo ip netns exec agent-4 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-4-eth0
+ create_agent 4 192.168.50.4 2 2
+ IDX=4
+ ADDR=192.168.50.4
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-4
+ echo Create agent agent-4
Create agent agent-4
+ sudo ip netns exec agent-4 ovsdb-tool create /tmp/skydive-scale/agent-4.db
+ sudo ip netns exec agent-4 ovsdb-server /tmp/skydive-scale/agent-4.db --remote=punix:/tmp/skydive-scale/agent-4.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-4-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-4-dbserver.pid --detach
2019-05-15T16:04:53Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-4-vswitchd.log
+ sudo ip netns exec agent-4 ovs-vswitchd unix:/tmp/skydive-scale/agent-4.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-4-vswitchd.log --pidfile=/tmp/skydive-scale/agent-4-vswitchd.pid --detach --monitor
+ mkdir -p /tmp/skydive-scale/agent-4-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-4.cover
+ PIDFILE=/tmp/skydive-scale/agent-4.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 6 ip netns exec agent-4 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-4.cover ; export PIDFILE=/tmp/skydive-scale/agent-4.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-4.yml 2>&1 | tee /tmp/skydive-scale/agent-4.log'
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-br agent-4
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock set bridge agent-4 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-4-vm1
Create VM agent-4-vm1
+ sudo ip netns add agent-4-vm1
+ sudo ip netns exec agent-4-vm1 ip l set lo up
+ sudo ip netns exec agent-4 ip link add vm1-eth0 type veth peer name eth0 netns agent-4-vm1
+ sudo ip netns exec agent-4 ip link set vm1-eth0 up
+ sudo ip netns exec agent-4-vm1 ip link set eth0 up
+ sudo ip netns exec agent-4-vm1 ip address add 10.0.4.1/16 dev eth0
+ sudo ip netns exec agent-4-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 vm1-eth0
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-4-vm2
Create VM agent-4-vm2
+ sudo ip netns add agent-4-vm2
+ sudo ip netns exec agent-4-vm2 ip l set lo up
+ sudo ip netns exec agent-4 ip link add vm2-eth0 type veth peer name eth0 netns agent-4-vm2
+ sudo ip netns exec agent-4 ip link set vm2-eth0 up
+ sudo ip netns exec agent-4-vm2 ip link set eth0 up
+ sudo ip netns exec agent-4-vm2 ip address add 10.0.4.2/16 dev eth0
+ sudo ip netns exec agent-4-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 vm2-eth0
+ touch /tmp/skydive-scale/agent-4.lock
+ '[' -z 192.168.50.1,192.168.50.2,192.168.50.3 ']'
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ WINDOW=7
++ seq 4
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-1 192.168.50.1 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ NAME=agent-1
+ ADDR=192.168.50.1
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.1 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.2 ']'
+ echo '192.168.50.1 -> 192.168.50.2'
192.168.50.1 -> 192.168.50.2
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.3 ']'
+ echo '192.168.50.1 -> 192.168.50.3'
192.168.50.1 -> 192.168.50.3
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.4 ']'
+ echo '192.168.50.1 -> 192.168.50.4'
192.168.50.1 -> 192.168.50.4
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.4 -- set interface gre-192.168.50.4 type=gre options:remote_ip=192.168.50.4
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-2 192.168.50.2 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ NAME=agent-2
+ ADDR=192.168.50.2
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ echo Create tunnels...
Create tunnels...
++ sed 's/,/ /g'
++ echo 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.1 ']'
+ echo '192.168.50.2 -> 192.168.50.1'
192.168.50.2 -> 192.168.50.1
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.2 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.3 ']'
+ echo '192.168.50.2 -> 192.168.50.3'
192.168.50.2 -> 192.168.50.3
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.4 ']'
+ echo '192.168.50.2 -> 192.168.50.4'
192.168.50.2 -> 192.168.50.4
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.4 -- set interface gre-192.168.50.4 type=gre options:remote_ip=192.168.50.4
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-3 192.168.50.3 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ NAME=agent-3
+ ADDR=192.168.50.3
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ echo Create tunnels...
Create tunnels...
++ sed 's/,/ /g'
++ echo 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.1 ']'
+ echo '192.168.50.3 -> 192.168.50.1'
192.168.50.3 -> 192.168.50.1
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.2 ']'
+ echo '192.168.50.3 -> 192.168.50.2'
192.168.50.3 -> 192.168.50.2
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.3 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.4 ']'
+ echo '192.168.50.3 -> 192.168.50.4'
192.168.50.3 -> 192.168.50.4
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.4 -- set interface gre-192.168.50.4 type=gre options:remote_ip=192.168.50.4
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-4 192.168.50.4 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ NAME=agent-4
+ ADDR=192.168.50.4
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.4 '!=' 192.168.50.1 ']'
+ echo '192.168.50.4 -> 192.168.50.1'
192.168.50.4 -> 192.168.50.1
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.4 '!=' 192.168.50.2 ']'
+ echo '192.168.50.4 -> 192.168.50.2'
192.168.50.4 -> 192.168.50.2
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.4 '!=' 192.168.50.3 ']'
+ echo '192.168.50.4 -> 192.168.50.3'
192.168.50.4 -> 192.168.50.3
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.4 '!=' 192.168.50.4 ']'
+ echo Go to : http://192.168.50.254:8082
Go to : http://192.168.50.254:8082

2019-05-15T16:04:57.396Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh stop-agent 4]
2019-05-15T16:04:57.759Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop-agent == start ']'
+ '[' stop-agent == stop ']'
+ '[' stop-agent == stop-analyzer ']'
+ '[' stop-agent == stop-agent ']'
+ '[' -z 4 ']'
+ stop_agent 4
+ AGENT=4
+ NAME=agent-4
+ '[' -e /tmp/skydive-scale/agent-4.pid ']'
+ sudo pkill -f agent-4.yml
+ timeout=30
+ pkill -0 -f agent-4.yml
+ sudo rm /tmp/skydive-scale/agent-4.lock /tmp/skydive-scale/agent-4.pid
rm: cannot remove '/tmp/skydive-scale/agent-4.pid': No such file or directory

2019-05-15T16:04:58.070Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh stop-analyzer 2]
2019-05-15T16:04:58.335Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop-analyzer == start ']'
+ '[' stop-analyzer == stop ']'
+ '[' stop-analyzer == stop-analyzer ']'
+ '[' -z 2 ']'
+ stop_analyzer 2
+ ANALYZER=2
+ NAME=analyzer-2
+ '[' -e /tmp/skydive-scale/analyzer-2.pid ']'
+ sudo pkill -f analyzer-2.yml
+ timeout=30
+ pkill -0 -f analyzer-2.yml
+ sudo rm /tmp/skydive-scale/analyzer-2.lock /tmp/skydive-scale/analyzer-2.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-2.pid': No such file or directory

2019-05-15T16:04:58.336Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [sleep 5]
2019-05-15T16:05:03.339Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned 
2019-05-15T16:05:03.429Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:03.579Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=3.05 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.054/3.054/3.054/0.000 ms

2019-05-15T16:05:04.579Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:04.731Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.258 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.258/0.258/0.258/0.000 ms

2019-05-15T16:05:05.731Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:05.877Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
++ sudo ip netns exec agent-1-vm1 ip a show eth0
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.157 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.157/0.157/0.157/0.000 ms

2019-05-15T16:05:06.877Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:06.998Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.250 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.250/0.250/0.250/0.000 ms

2019-05-15T16:05:08.000Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:08.148Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.228 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.228/0.228/0.228/0.000 ms

2019-05-15T16:05:09.148Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:09.279Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.188 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.188/0.188/0.188/0.000 ms

2019-05-15T16:05:10.280Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:10.415Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.193 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.193/0.193/0.193/0.000 ms

2019-05-15T16:05:11.416Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:11.531Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.159 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.159/0.159/0.159/0.000 ms

2019-05-15T16:05:12.532Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:12.655Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.135 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.135/0.135/0.135/0.000 ms

2019-05-15T16:05:13.655Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:13.812Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.163 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.163/0.163/0.163/0.000 ms

2019-05-15T16:05:14.812Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:14.970Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.246 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.246/0.246/0.246/0.000 ms

2019-05-15T16:05:15.971Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:16.096Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.215 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.215/0.215/0.215/0.000 ms

2019-05-15T16:05:17.097Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:17.210Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.155 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.155/0.155/0.155/0.000 ms

2019-05-15T16:05:18.211Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:18.346Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.135 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.135/0.135/0.135/0.000 ms

2019-05-15T16:05:19.347Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:19.468Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.173 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.173/0.173/0.173/0.000 ms

2019-05-15T16:05:20.469Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:20.583Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.215 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.215/0.215/0.215/0.000 ms

2019-05-15T16:05:21.584Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:21.699Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.125 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.125/0.125/0.125/0.000 ms

2019-05-15T16:05:22.699Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:22.844Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.137 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.137/0.137/0.137/0.000 ms

2019-05-15T16:05:23.844Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:23.972Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.184 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.184/0.184/0.184/0.000 ms

2019-05-15T16:05:24.972Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:25.104Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.222 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.222/0.222/0.222/0.000 ms

2019-05-15T16:05:26.104Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:26.243Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.138 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.138/0.138/0.138/0.000 ms

2019-05-15T16:05:27.244Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:27.388Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.170 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.170/0.170/0.170/0.000 ms

2019-05-15T16:05:28.388Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:28.508Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ cut -d / -f 1
++ grep 'inet '
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.176 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.176/0.176/0.176/0.000 ms

2019-05-15T16:05:29.509Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:29.622Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.196 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.196/0.196/0.196/0.000 ms

2019-05-15T16:05:30.623Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:30.758Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.205 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.205/0.205/0.205/0.000 ms

2019-05-15T16:05:31.758Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:31.890Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.137 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.137/0.137/0.137/0.000 ms

2019-05-15T16:05:32.891Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:33.008Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.190 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.190/0.190/0.190/0.000 ms

2019-05-15T16:05:34.008Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:34.145Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.261 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.261/0.261/0.261/0.000 ms

2019-05-15T16:05:35.146Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:35.276Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.214 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.214/0.214/0.214/0.000 ms

2019-05-15T16:05:36.276Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:05:36.397Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.150 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.150/0.150/0.150/0.000 ms

2019-05-15T16:05:46.654Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh iperf agent-3-vm1 agent-1-vm1]
2019-05-15T16:05:59.354Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' iperf == start ']'
+ '[' iperf == stop ']'
+ '[' iperf == stop-analyzer ']'
+ '[' iperf == stop-agent ']'
+ '[' iperf == ping ']'
+ '[' iperf == iperf ']'
+ '[' -z agent-1-vm1 ']'
+ iperf agent-3-vm1 agent-1-vm1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS=
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sleep 1
+ sudo ip netns exec agent-1-vm1 iperf -s -t 11
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------
+ sudo ip netns exec agent-3-vm1 iperf -c 10.0.1.1 -t 10 -b 1M
------------------------------------------------------------
Client connecting to 10.0.1.1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.3.1 port 34810 connected with 10.0.1.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.2 sec   107 KBytes  86.5 Kbits/sec
+ sudo pkill -f 'iperf -s'
+ sleep 1
+ sudo pkill -f 'iperf -s'

2019-05-15T16:06:02.484Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh start 2 3 2]
2019-05-15T16:06:03.658Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' start == start ']'
+ '[' -z 2 ']'
+ start 2 3 2
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ VM_NUM=2
+ '[' '!' -e /tmp/skydive-scale/screen.lock ']'
+ '[' true = true ']'
+ generate_ssl_conf 2 3
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ cat
++ seq 2
+ for I in $( seq $ANALYZER_NUM )
+ IDX=4
+ echo IP.4 = 192.168.50.1
+ for I in $( seq $ANALYZER_NUM )
+ IDX=5
+ echo IP.5 = 192.168.50.2
+ TOTAL_AGENT=8
++ seq 8
+ for I in $( seq $TOTAL_AGENT )
+ IP=3
+ IDX=5
+ echo IP.5 = 192.168.50.3
+ for I in $( seq $TOTAL_AGENT )
+ IP=4
+ IDX=6
+ echo IP.6 = 192.168.50.4
+ for I in $( seq $TOTAL_AGENT )
+ IP=5
+ IDX=7
+ echo IP.7 = 192.168.50.5
+ for I in $( seq $TOTAL_AGENT )
+ IP=6
+ IDX=8
+ echo IP.8 = 192.168.50.6
+ for I in $( seq $TOTAL_AGENT )
+ IP=7
+ IDX=9
+ echo IP.9 = 192.168.50.7
+ for I in $( seq $TOTAL_AGENT )
+ IP=8
+ IDX=10
+ echo IP.10 = 192.168.50.8
+ for I in $( seq $TOTAL_AGENT )
+ IP=9
+ IDX=11
+ echo IP.11 = 192.168.50.9
+ for I in $( seq $TOTAL_AGENT )
+ IP=10
+ IDX=12
+ echo IP.12 = 192.168.50.10
+ generate_tls_crt analyzer
+ NAME=analyzer
+ '[' -e /tmp/skydive-scale/analyzer.crt ']'
+ return
+ generate_tls_crt agent
+ NAME=agent
+ '[' -e /tmp/skydive-scale/agent.crt ']'
+ return
+ create_main_bridge
+ echo Create central bridge
Create central bridge
+ sudo brctl addbr br-central
device br-central already exists; can't create bridge with the same name
+ sudo brctl stp br-central off
+ sudo ip l set br-central up
+ sudo ip a add 192.168.50.254/24 dev br-central
RTNETLINK answers: File exists
+ start_inotify
+ '[' -f /tmp/skydive-scale/inotify.lock ']'
+ return
+ WINDOW=1
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 1 -gt 1 ']'
+ '[' '!' -f /tmp/skydive-scale/analyzer-1.lock ']'
+ CURR_ANALYZER_PORT=8084
+ WINDOW=2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 2 -gt 1 ']'
+ ETCD_EMBEDDED=false
+ '[' '!' -f /tmp/skydive-scale/analyzer-2.lock ']'
+ create_analyzer 2 2 3
+ IDX=2
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ NAME=analyzer-2
+ echo Create analyzer analyzer-2
Create analyzer analyzer-2
+ mkdir -p /tmp/skydive-scale/analyzer-2-etcd
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - localhost:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - localhost:8084'
+ STORAGE=
+ GRAPH=memory
+ '[' xlocalhost:9200 '!=' x ']'
+ STORAGE=elasticsearch
+ GRAPH=elasticsearch
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ cat
+ TOTAL_AGENT=8
++ seq 8
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_1'
+ echo '      - TOR_PORT_1 --> *[Name=agent-1]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_2'
+ echo '      - TOR_PORT_2 --> *[Name=agent-2]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_3'
+ echo '      - TOR_PORT_3 --> *[Name=agent-3]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_4'
+ echo '      - TOR_PORT_4 --> *[Name=agent-4]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_5'
+ echo '      - TOR_PORT_5 --> *[Name=agent-5]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_6'
+ echo '      - TOR_PORT_6 --> *[Name=agent-6]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_7'
+ echo '      - TOR_PORT_7 --> *[Name=agent-7]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_8'
+ echo '      - TOR_PORT_8 --> *[Name=agent-8]/eth0'
+ cat
++ seq 8
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-1: agent-password-1'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-2: agent-password-2'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-3: agent-password-3'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-4: agent-password-4'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-5: agent-password-5'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-6: agent-password-6'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-7: agent-password-7'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-8: agent-password-8'
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-1: analyzer-password-1'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-2: analyzer-password-2'
+ COVERFILE=/tmp/skydive-scale/analyzer-2.cover
+ PIDFILE=/tmp/skydive-scale/analyzer-2.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-analyzer 2 sh -c 'export COVERFILE=/tmp/skydive-scale/analyzer-2.cover ; export PIDFILE=/tmp/skydive-scale/analyzer-2.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive analyzer -c /tmp/skydive-scale/analyzer-2.yml 2>&1 | tee /tmp/skydive-scale/analyzer-2.log'
+ touch /tmp/skydive-scale/analyzer-2.lock
+ CURR_ANALYZER_PORT=8086
+ WINDOW=3
+ AGENTS=
++ seq 3
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-1.lock ']'
+ '[' -z '' ']'
+ AGENTS=192.168.50.1
+ WINDOW=4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-2.lock ']'
+ '[' -z 192.168.50.1 ']'
+ AGENTS=192.168.50.1,192.168.50.2
+ WINDOW=5
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-3.lock ']'
+ '[' -z 192.168.50.1,192.168.50.2 ']'
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ WINDOW=6
++ seq 3
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-1 192.168.50.1 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-1
+ ADDR=192.168.50.1
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ sed 's/,/ /g'
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.1 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.2 ']'
+ echo '192.168.50.1 -> 192.168.50.2'
192.168.50.1 -> 192.168.50.2
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.3 ']'
+ echo '192.168.50.1 -> 192.168.50.3'
192.168.50.1 -> 192.168.50.3
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
ovs-vsctl: cannot create a port named gre-192.168.50.3 because a port named gre-192.168.50.3 already exists on bridge agent-1
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-2 192.168.50.2 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-2
+ ADDR=192.168.50.2
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.1 ']'
+ echo '192.168.50.2 -> 192.168.50.1'
192.168.50.2 -> 192.168.50.1
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.2 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.3 ']'
+ echo '192.168.50.2 -> 192.168.50.3'
192.168.50.2 -> 192.168.50.3
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
ovs-vsctl: cannot create a port named gre-192.168.50.3 because a port named gre-192.168.50.3 already exists on bridge agent-2
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-3 192.168.50.3 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-3
+ ADDR=192.168.50.3
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.1 ']'
+ echo '192.168.50.3 -> 192.168.50.1'
192.168.50.3 -> 192.168.50.1
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.2 ']'
+ echo '192.168.50.3 -> 192.168.50.2'
192.168.50.3 -> 192.168.50.2
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.3 ']'
+ echo Go to : http://192.168.50.254:8082
Go to : http://192.168.50.254:8082

2019-05-15T16:06:03.658Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [sleep 5]
2019-05-15T16:06:08.660Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned 
2019-05-15T16:06:08.665Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh stop-agent 1]
2019-05-15T16:06:08.957Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop-agent == start ']'
+ '[' stop-agent == stop ']'
+ '[' stop-agent == stop-analyzer ']'
+ '[' stop-agent == stop-agent ']'
+ '[' -z 1 ']'
+ stop_agent 1
+ AGENT=1
+ NAME=agent-1
+ '[' -e /tmp/skydive-scale/agent-1.pid ']'
+ sudo pkill -f agent-1.yml
+ timeout=30
+ pkill -0 -f agent-1.yml
+ sudo rm /tmp/skydive-scale/agent-1.lock /tmp/skydive-scale/agent-1.pid
rm: cannot remove '/tmp/skydive-scale/agent-1.pid': No such file or directory

2019-05-15T16:06:09.105Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh start 2 3 2]
2019-05-15T16:06:11.714Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' start == start ']'
+ '[' -z 2 ']'
+ start 2 3 2
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ VM_NUM=2
+ '[' '!' -e /tmp/skydive-scale/screen.lock ']'
+ '[' true = true ']'
+ generate_ssl_conf 2 3
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ cat
++ seq 2
+ for I in $( seq $ANALYZER_NUM )
+ IDX=4
+ echo IP.4 = 192.168.50.1
+ for I in $( seq $ANALYZER_NUM )
+ IDX=5
+ echo IP.5 = 192.168.50.2
+ TOTAL_AGENT=8
++ seq 8
+ for I in $( seq $TOTAL_AGENT )
+ IP=3
+ IDX=5
+ echo IP.5 = 192.168.50.3
+ for I in $( seq $TOTAL_AGENT )
+ IP=4
+ IDX=6
+ echo IP.6 = 192.168.50.4
+ for I in $( seq $TOTAL_AGENT )
+ IP=5
+ IDX=7
+ echo IP.7 = 192.168.50.5
+ for I in $( seq $TOTAL_AGENT )
+ IP=6
+ IDX=8
+ echo IP.8 = 192.168.50.6
+ for I in $( seq $TOTAL_AGENT )
+ IP=7
+ IDX=9
+ echo IP.9 = 192.168.50.7
+ for I in $( seq $TOTAL_AGENT )
+ IP=8
+ IDX=10
+ echo IP.10 = 192.168.50.8
+ for I in $( seq $TOTAL_AGENT )
+ IP=9
+ IDX=11
+ echo IP.11 = 192.168.50.9
+ for I in $( seq $TOTAL_AGENT )
+ IP=10
+ IDX=12
+ echo IP.12 = 192.168.50.10
+ generate_tls_crt analyzer
+ NAME=analyzer
+ '[' -e /tmp/skydive-scale/analyzer.crt ']'
+ return
+ generate_tls_crt agent
+ NAME=agent
+ '[' -e /tmp/skydive-scale/agent.crt ']'
+ return
+ create_main_bridge
+ echo Create central bridge
Create central bridge
+ sudo brctl addbr br-central
device br-central already exists; can't create bridge with the same name
+ sudo brctl stp br-central off
+ sudo ip l set br-central up
+ sudo ip a add 192.168.50.254/24 dev br-central
RTNETLINK answers: File exists
+ start_inotify
+ '[' -f /tmp/skydive-scale/inotify.lock ']'
+ return
+ WINDOW=1
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 1 -gt 1 ']'
+ '[' '!' -f /tmp/skydive-scale/analyzer-1.lock ']'
+ CURR_ANALYZER_PORT=8084
+ WINDOW=2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 2 -gt 1 ']'
+ ETCD_EMBEDDED=false
+ '[' '!' -f /tmp/skydive-scale/analyzer-2.lock ']'
+ CURR_ANALYZER_PORT=8086
+ WINDOW=3
+ AGENTS=
++ seq 3
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-1.lock ']'
+ create_host agent-1 192.168.50.1
+ NAME=agent-1
+ ADDR=192.168.50.1
+ echo Create host agent-1
Create host agent-1
+ sudo ip netns add agent-1
Cannot create namespace file "/var/run/netns/agent-1": File exists
+ sudo ip netns exec agent-1 ip link set lo up
+ sudo ip link add agent-1-eth0 type veth peer name eth0 netns agent-1
RTNETLINK answers: File exists
+ sudo ip link set agent-1-eth0 up
+ sudo ip netns exec agent-1 ip link set eth0 up
+ sudo ip netns exec agent-1 ip address add 192.168.50.1/24 dev eth0
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1 ip route add 0.0.0.0/0 via 192.168.50.254
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-1-eth0
device agent-1-eth0 is already a member of a bridge; can't enslave it to bridge br-central.
+ create_agent 1 192.168.50.1 2 2
+ IDX=1
+ ADDR=192.168.50.1
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-1
+ echo Create agent agent-1
Create agent agent-1
+ sudo ip netns exec agent-1 ovsdb-tool create /tmp/skydive-scale/agent-1.db
2019-05-15T16:06:09Z|00001|lockfile|WARN|/tmp/skydive-scale/.agent-1.db.~lock~: cannot lock file because it is already locked by pid 13728
ovsdb-tool: I/O error: /tmp/skydive-scale/agent-1.db: failed to lock lockfile (Resource temporarily unavailable)
+ sudo ip netns exec agent-1 ovsdb-server /tmp/skydive-scale/agent-1.db --remote=punix:/tmp/skydive-scale/agent-1.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-1-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-1-dbserver.pid --detach
2019-05-15T16:06:09Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-1-vswitchd.log
ovsdb-server: /tmp/skydive-scale/agent-1-dbserver.pid: already running as pid 13728, aborting
+ sudo ip netns exec agent-1 ovs-vswitchd unix:/tmp/skydive-scale/agent-1.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-1-vswitchd.log --pidfile=/tmp/skydive-scale/agent-1-vswitchd.pid --detach --monitor
ovs-vswitchd: /tmp/skydive-scale/agent-1-vswitchd.pid: already running as pid 13733, aborting
+ mkdir -p /tmp/skydive-scale/agent-1-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-1.cover
+ PIDFILE=/tmp/skydive-scale/agent-1.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 3 ip netns exec agent-1 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-1.cover ; export PIDFILE=/tmp/skydive-scale/agent-1.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-1.yml 2>&1 | tee /tmp/skydive-scale/agent-1.log'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-br agent-1
ovs-vsctl: cannot create a bridge named agent-1 because a bridge named agent-1 already exists
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock set bridge agent-1 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-1-vm1
Create VM agent-1-vm1
+ sudo ip netns add agent-1-vm1
Cannot create namespace file "/var/run/netns/agent-1-vm1": File exists
+ sudo ip netns exec agent-1-vm1 ip l set lo up
+ sudo ip netns exec agent-1 ip link add vm1-eth0 type veth peer name eth0 netns agent-1-vm1
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1 ip link set vm1-eth0 up
+ sudo ip netns exec agent-1-vm1 ip link set eth0 up
+ sudo ip netns exec agent-1-vm1 ip address add 10.0.1.1/16 dev eth0
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 vm1-eth0
ovs-vsctl: cannot create a port named vm1-eth0 because a port named vm1-eth0 already exists on bridge agent-1
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-1-vm2
Create VM agent-1-vm2
+ sudo ip netns add agent-1-vm2
Cannot create namespace file "/var/run/netns/agent-1-vm2": File exists
+ sudo ip netns exec agent-1-vm2 ip l set lo up
+ sudo ip netns exec agent-1 ip link add vm2-eth0 type veth peer name eth0 netns agent-1-vm2
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1 ip link set vm2-eth0 up
+ sudo ip netns exec agent-1-vm2 ip link set eth0 up
+ sudo ip netns exec agent-1-vm2 ip address add 10.0.1.2/16 dev eth0
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 vm2-eth0
ovs-vsctl: cannot create a port named vm2-eth0 because a port named vm2-eth0 already exists on bridge agent-1
+ touch /tmp/skydive-scale/agent-1.lock
+ '[' -z '' ']'
+ AGENTS=192.168.50.1
+ WINDOW=4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-2.lock ']'
+ '[' -z 192.168.50.1 ']'
+ AGENTS=192.168.50.1,192.168.50.2
+ WINDOW=5
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-3.lock ']'
+ '[' -z 192.168.50.1,192.168.50.2 ']'
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ WINDOW=6
++ seq 3
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-1 192.168.50.1 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-1
+ ADDR=192.168.50.1
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.1 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.2 ']'
+ echo '192.168.50.1 -> 192.168.50.2'
192.168.50.1 -> 192.168.50.2
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.3 ']'
+ echo '192.168.50.1 -> 192.168.50.3'
192.168.50.1 -> 192.168.50.3
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
ovs-vsctl: cannot create a port named gre-192.168.50.3 because a port named gre-192.168.50.3 already exists on bridge agent-1
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-2 192.168.50.2 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-2
+ ADDR=192.168.50.2
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.1 ']'
+ echo '192.168.50.2 -> 192.168.50.1'
192.168.50.2 -> 192.168.50.1
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.2 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.3 ']'
+ echo '192.168.50.2 -> 192.168.50.3'
192.168.50.2 -> 192.168.50.3
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
ovs-vsctl: cannot create a port named gre-192.168.50.3 because a port named gre-192.168.50.3 already exists on bridge agent-2
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-3 192.168.50.3 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-3
+ ADDR=192.168.50.3
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.1 ']'
+ echo '192.168.50.3 -> 192.168.50.1'
192.168.50.3 -> 192.168.50.1
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.2 ']'
+ echo '192.168.50.3 -> 192.168.50.2'
192.168.50.3 -> 192.168.50.2
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.3 ']'
+ echo Go to : http://192.168.50.254:8082
Go to : http://192.168.50.254:8082

2019-05-15T16:06:11.715Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [sleep 5]
2019-05-15T16:06:16.718Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned 
2019-05-15T16:06:16.942Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:17.077Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=1.14 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.138/1.138/1.138/0.000 ms

2019-05-15T16:06:18.077Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:18.218Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.198 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.198/0.198/0.198/0.000 ms

2019-05-15T16:06:19.218Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:19.346Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.130 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.130/0.130/0.130/0.000 ms

2019-05-15T16:06:20.347Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:20.462Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.120 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.120/0.120/0.120/0.000 ms

2019-05-15T16:06:21.462Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:21.584Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.130 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.130/0.130/0.130/0.000 ms

2019-05-15T16:06:22.585Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:22.701Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.192 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.192/0.192/0.192/0.000 ms

2019-05-15T16:06:23.701Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:23.848Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ cut -d / -f 1
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.137 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.137/0.137/0.137/0.000 ms

2019-05-15T16:06:24.848Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:24.984Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.174 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.174/0.174/0.174/0.000 ms

2019-05-15T16:06:25.984Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:26.132Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.142 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.142/0.142/0.142/0.000 ms

2019-05-15T16:06:27.132Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:27.253Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.133 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.133/0.133/0.133/0.000 ms

2019-05-15T16:06:28.253Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:28.385Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.176 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.176/0.176/0.176/0.000 ms

2019-05-15T16:06:29.385Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:29.512Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.101 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.101/0.101/0.101/0.000 ms

2019-05-15T16:06:30.513Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:30.627Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.178 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.178/0.178/0.178/0.000 ms

2019-05-15T16:06:31.628Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:31.783Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
++ sudo ip netns exec agent-2-vm1 ip a show eth0
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.157 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.157/0.157/0.157/0.000 ms

2019-05-15T16:06:32.783Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:32.936Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.162 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.162/0.162/0.162/0.000 ms

2019-05-15T16:06:33.936Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:34.062Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.132 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.132/0.132/0.132/0.000 ms

2019-05-15T16:06:35.062Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:35.198Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.156 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.156/0.156/0.156/0.000 ms

2019-05-15T16:06:36.199Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:36.372Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.132 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.132/0.132/0.132/0.000 ms

2019-05-15T16:06:37.373Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:37.500Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.124 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.124/0.124/0.124/0.000 ms

2019-05-15T16:06:38.501Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:38.638Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.155 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.155/0.155/0.155/0.000 ms

2019-05-15T16:06:39.639Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:39.761Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.123 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.123/0.123/0.123/0.000 ms

2019-05-15T16:06:40.761Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:40.922Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.138 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.138/0.138/0.138/0.000 ms

2019-05-15T16:06:41.923Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:42.054Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.126 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.126/0.126/0.126/0.000 ms

2019-05-15T16:06:43.055Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:43.239Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ grep 'inet '
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.196 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.196/0.196/0.196/0.000 ms

2019-05-15T16:06:44.239Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:44.373Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.131 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.131/0.131/0.131/0.000 ms

2019-05-15T16:06:45.373Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:45.510Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.115 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.115/0.115/0.115/0.000 ms

2019-05-15T16:06:46.510Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:46.629Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.148 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.148/0.148/0.148/0.000 ms

2019-05-15T16:06:47.629Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:47.779Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.113 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.113/0.113/0.113/0.000 ms

2019-05-15T16:06:48.780Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:48.946Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.135 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.135/0.135/0.135/0.000 ms

2019-05-15T16:06:49.946Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:06:50.101Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.137 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.137/0.137/0.137/0.000 ms

2019-05-15T16:06:51.587Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh stop 2 4 2]
2019-05-15T16:06:59.946Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop == start ']'
+ '[' stop == stop ']'
+ '[' -z 2 ']'
+ stop 2 4 2
+ ANALYZER_NUM=2
+ AGENT_NUM=4
+ VM_NUM=2
+ echo 'Stopping Agents...'
Stopping Agents...
++ seq 4
+ for A in $( seq $AGENT_NUM )
+ stop_agent 1
+ AGENT=1
+ NAME=agent-1
+ '[' -e /tmp/skydive-scale/agent-1.pid ']'
+ sudo pkill -f agent-1.yml
+ timeout=30
+ pkill -0 -f agent-1.yml
+ sudo rm /tmp/skydive-scale/agent-1.lock /tmp/skydive-scale/agent-1.pid
rm: cannot remove '/tmp/skydive-scale/agent-1.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 2
+ AGENT=2
+ NAME=agent-2
+ '[' -e /tmp/skydive-scale/agent-2.pid ']'
+ sudo pkill -f agent-2.yml
+ timeout=30
+ pkill -0 -f agent-2.yml
+ sudo rm /tmp/skydive-scale/agent-2.lock /tmp/skydive-scale/agent-2.pid
rm: cannot remove '/tmp/skydive-scale/agent-2.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 3
+ AGENT=3
+ NAME=agent-3
+ '[' -e /tmp/skydive-scale/agent-3.pid ']'
+ sudo pkill -f agent-3.yml
+ timeout=30
+ pkill -0 -f agent-3.yml
+ sudo rm /tmp/skydive-scale/agent-3.lock /tmp/skydive-scale/agent-3.pid
rm: cannot remove '/tmp/skydive-scale/agent-3.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 4
+ AGENT=4
+ NAME=agent-4
+ '[' -e /tmp/skydive-scale/agent-4.pid ']'
+ sudo pkill -f agent-4.yml
+ timeout=30
+ pkill -0 -f agent-4.yml
+ sudo rm /tmp/skydive-scale/agent-4.lock /tmp/skydive-scale/agent-4.pid
rm: cannot remove '/tmp/skydive-scale/agent-4.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-4.pid': No such file or directory
+ echo 'Stopping Analyzers...'
Stopping Analyzers...
++ seq 2
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 1
+ ANALYZER=1
+ NAME=analyzer-1
+ '[' -e /tmp/skydive-scale/analyzer-1.pid ']'
+ sudo pkill -f analyzer-1.yml
+ timeout=30
+ pkill -0 -f analyzer-1.yml
+ sudo rm /tmp/skydive-scale/analyzer-1.lock /tmp/skydive-scale/analyzer-1.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-1.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 2
+ ANALYZER=2
+ NAME=analyzer-2
+ '[' -e /tmp/skydive-scale/analyzer-2.pid ']'
+ sudo pkill -f analyzer-2.yml
+ timeout=30
+ pkill -0 -f analyzer-2.yml
+ sudo rm /tmp/skydive-scale/analyzer-2.lock /tmp/skydive-scale/analyzer-2.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-2.pid': No such file or directory
+ sleep 5
+ sudo -E screen -S skydive-stress -X quit
+ rm /tmp/skydive-scale/screen.lock
++ seq 4
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 1 2
+ IDX=1
+ VM_NUM=2
+ NAME=agent-1
+ echo Delete agent agent-1
Delete agent agent-1
++ cat /tmp/skydive-scale/agent-1-vswitchd.pid
+ sudo kill -9 13733
++ cat /tmp/skydive-scale/agent-1-dbserver.pid
+ sudo kill -9 13728
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm1
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm2
+ rm /tmp/skydive-scale/agent-1.lock
rm: cannot remove '/tmp/skydive-scale/agent-1.lock': No such file or directory
+ delete_host agent-1
+ NAME=agent-1
+ echo Delete Host agent-1
Delete Host agent-1
+ sudo ovs-vsctl del-port br-central agent-1-eth0
ovs-vsctl: no port named agent-1-eth0
+ sudo ip link del agent-1-eth0
+ sudo ip netns del agent-1
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 2 2
+ IDX=2
+ VM_NUM=2
+ NAME=agent-2
+ echo Delete agent agent-2
Delete agent agent-2
++ cat /tmp/skydive-scale/agent-2-vswitchd.pid
+ sudo kill -9 13921
++ cat /tmp/skydive-scale/agent-2-dbserver.pid
+ sudo kill -9 13916
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm1
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm2
+ rm /tmp/skydive-scale/agent-2.lock
rm: cannot remove '/tmp/skydive-scale/agent-2.lock': No such file or directory
+ delete_host agent-2
+ NAME=agent-2
+ echo Delete Host agent-2
Delete Host agent-2
+ sudo ovs-vsctl del-port br-central agent-2-eth0
ovs-vsctl: no port named agent-2-eth0
+ sudo ip link del agent-2-eth0
+ sudo ip netns del agent-2
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 3 2
+ IDX=3
+ VM_NUM=2
+ NAME=agent-3
+ echo Delete agent agent-3
Delete agent agent-3
++ cat /tmp/skydive-scale/agent-3-vswitchd.pid
+ sudo kill -9 14710
++ cat /tmp/skydive-scale/agent-3-dbserver.pid
+ sudo kill -9 14705
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm1
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm2
+ rm /tmp/skydive-scale/agent-3.lock
rm: cannot remove '/tmp/skydive-scale/agent-3.lock': No such file or directory
+ delete_host agent-3
+ NAME=agent-3
+ echo Delete Host agent-3
Delete Host agent-3
+ sudo ovs-vsctl del-port br-central agent-3-eth0
ovs-vsctl: no port named agent-3-eth0
+ sudo ip link del agent-3-eth0
+ sudo ip netns del agent-3
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 4 2
+ IDX=4
+ VM_NUM=2
+ NAME=agent-4
+ echo Delete agent agent-4
Delete agent agent-4
++ cat /tmp/skydive-scale/agent-4-vswitchd.pid
+ sudo kill -9 14883
++ cat /tmp/skydive-scale/agent-4-dbserver.pid
+ sudo kill -9 14877
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm1
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm2
+ rm /tmp/skydive-scale/agent-4.lock
rm: cannot remove '/tmp/skydive-scale/agent-4.lock': No such file or directory
+ delete_host agent-4
+ NAME=agent-4
+ echo Delete Host agent-4
Delete Host agent-4
+ sudo ovs-vsctl del-port br-central agent-4-eth0
ovs-vsctl: no port named agent-4-eth0
+ sudo ip link del agent-4-eth0
+ sudo ip netns del agent-4
+ delete_main_bridge
+ echo Delete central bridge
Delete central bridge
+ sudo ip link del br-central
+ sudo find /tmp/skydive-scale -mindepth 1 '!' -name '*.log' -a '!' -name '*.cover' -exec rm -rf '{}' ';'
find: ‘/tmp/skydive-scale/agent-4-netns’: No such file or directory
find: ‘/tmp/skydive-scale/agent-1-netns’: No such file or directory
find: ‘/tmp/skydive-scale/analyzer-2-etcd’: No such file or directory
find: ‘/tmp/skydive-scale/agent-3-netns’: No such file or directory
find: ‘/tmp/skydive-scale/agent-2-netns’: No such file or directory
find: ‘/tmp/skydive-scale/analyzer-1-etcd’: No such file or directory

--- PASS: TestScaleHA (210.27s)
PASS
make[1]: Leaving directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive'
status=$?

cat /tmp/skydive-scale/{analyzer,agent}-?.log | sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" | perl -ne '$d=$1 if /^(\d+-\d+-\d+),/; $k{$d}.=$_; END{print $k{$_} for sort keys(%k);}'
2019-05-15T16:03:32.059Z	INFO	analyzer/analyzer.go:42 glob..func1	analyzer-1: Skydive Analyzer 0.22.0-5bebd28d8830 starting...
2019-05-15 16:03:32.060767 I | embed: listening for peers on http://192.168.50.254:12380
2019-05-15 16:03:32.060977 I | embed: listening for client requests on 192.168.50.254:12379
2019-05-15 16:03:32.064753 I | etcdserver: name = jenkins-slave-2.rdocloud
2019-05-15 16:03:32.064923 I | etcdserver: data dir = /tmp/skydive-scale/analyzer-1-etcd
2019-05-15 16:03:32.064934 I | etcdserver: member dir = /tmp/skydive-scale/analyzer-1-etcd/member
2019-05-15 16:03:32.064938 I | etcdserver: heartbeat = 100ms
2019-05-15 16:03:32.064955 I | etcdserver: election = 1000ms
2019-05-15 16:03:32.064959 I | etcdserver: snapshot count = 100000
2019-05-15 16:03:32.064992 I | etcdserver: advertise client URLs = http://192.168.50.254:12379
2019-05-15 16:03:32.064998 I | etcdserver: initial advertise peer URLs = http://localhost:12380
2019-05-15 16:03:32.065010 I | etcdserver: initial cluster = jenkins-slave-2.rdocloud=http://localhost:12380
2019-05-15 16:03:32.074773 I | etcdserver: starting member c9ac9fc89eae9cf7 in cluster f290f7f84a022555
2019-05-15 16:03:32.074920 I | raft: c9ac9fc89eae9cf7 became follower at term 0
2019-05-15 16:03:32.075006 I | raft: newRaft c9ac9fc89eae9cf7 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2019-05-15 16:03:32.075048 I | raft: c9ac9fc89eae9cf7 became follower at term 1
2019-05-15 16:03:32.081509 W | auth: simple token is not cryptographically signed
2019-05-15 16:03:32.082847 I | etcdserver: starting server... [version: 3.2.15+git, cluster version: to_be_decided]
2019-05-15 16:03:32.083582 I | etcdserver/membership: added member c9ac9fc89eae9cf7 [http://localhost:12380] to cluster f290f7f84a022555
2019-05-15 16:03:33.076906 I | raft: c9ac9fc89eae9cf7 is starting a new election at term 1
2019-05-15 16:03:33.076957 I | raft: c9ac9fc89eae9cf7 became candidate at term 2
2019-05-15 16:03:33.076988 I | raft: c9ac9fc89eae9cf7 received MsgVoteResp from c9ac9fc89eae9cf7 at term 2
2019-05-15 16:03:33.077002 I | raft: c9ac9fc89eae9cf7 became leader at term 2
2019-05-15 16:03:33.077016 I | raft: raft.node: c9ac9fc89eae9cf7 elected leader c9ac9fc89eae9cf7 at term 2
2019-05-15 16:03:33.077348 I | etcdserver: published {Name:jenkins-slave-2.rdocloud ClientURLs:[http://192.168.50.254:12379]} to cluster f290f7f84a022555
2019-05-15 16:03:33.077371 I | Server is ready!
2019-05-15 16:03:33.077403 I | embed: ready to serve client requests
2019-05-15 16:03:33.077475 I | etcdserver: setting up the initial cluster version to 3.2
2019-05-15 16:03:33.077866 N | embed: serving insecure client requests on 192.168.50.254:12379, this is strongly discouraged!
2019-05-15 16:03:33.078133 N | etcdserver/membership: set the initial cluster version to 3.2
2019-05-15 16:03:33.078186 I | etcdserver/api: enabled capabilities for version 3.2
2019-05-15T16:03:33.080Z	DEBUG	etcd/server.go:136 NewEmbeddedEtcd	analyzer-1: Successfully started etcd
2019-05-15T16:03:33.083Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Model:
2019-05-15T16:03:33.083Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Model:
2019-05-15T16:03:33.083Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: p.p: sub, obj, act, eft
2019-05-15T16:03:33.083Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: e.e: some(where (p_eft == allow)) && !some(where (p_eft == deny))
2019-05-15T16:03:33.083Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: m.m: g(r_sub, p_sub) && r_obj == p_obj && r_act == p_act
2019-05-15T16:03:33.083Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: g.g: _, _
2019-05-15T16:03:33.083Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: r.r: sub, obj, act
2019-05-15T16:03:33.084Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:03:33.084Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: 
2019-05-15T16:03:33.085Z	INFO	analyzer/storage.go:59 newGraphBackendFromConfig	analyzer-1: Using elasticsearch (driver elasticsearch) as graph storage backend
2019-05-15T16:03:33.089Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-es-graph-flush: analyzer-1
2019-05-15T16:03:34.516Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-es-rolling-index:1d8779e5: analyzer-1
2019-05-15T16:03:34.516Z	INFO	elasticsearch/client.go:225 (*Client).start	analyzer-1: client started for skydive_topology_live, skydive_topology_archive
2019-05-15T16:03:34.516Z	INFO	graph/elasticsearch.go:528 (*ElasticSearchBackend).flushGraph	analyzer-1: Flush graph elements
2019-05-15T16:03:34.727Z	INFO	analyzer/storage.go:85 newFlowBackendFromConfig	analyzer-1: Using elasticsearch (driver elasticsearch) as flow storage backend
2019-05-15T16:03:34.738Z	INFO	analyzer/flow_server.go:199 NewFlowServerUDPConn	analyzer-1: Analyzer listen agents on UDP socket
2019-05-15T16:03:34.834Z	INFO	http/server.go:104 (*Server).Listen	analyzer-1: Listening on socket 0.0.0.0:8082
2019-05-15T16:03:34.835Z	INFO	hub/replication_endpoint.go:140 (*ReplicatorPeer).connect	analyzer-1: Connecting to peer: wss://[::1]:8084/ws/replication
2019-05-15T16:03:34.835Z	INFO	hub/replication_endpoint.go:140 (*ReplicatorPeer).connect	analyzer-1: Connecting to peer: wss://[::1]:8082/ws/replication
2019-05-15T16:03:34.836Z	INFO	websocket/client.go:495 (*Client).Connect	analyzer-1: Connecting to wss://[::1]:8082/ws/replication
2019-05-15T16:03:34.836Z	INFO	websocket/client.go:495 (*Client).Connect	analyzer-1: Connecting to wss://[::1]:8084/ws/replication
2019-05-15T16:03:34.836Z	ERROR	websocket/client.go:557 (*Client).Start.func1	analyzer-1: Unable to create a WebSocket connection wss://[::1]:8084/ws/replication : dial tcp [::1]:8084: connect: connection refused
2019-05-15T16:03:34.844Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-ondemand-client: analyzer-1
2019-05-15T16:03:34.859Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-pi-client: analyzer-1
2019-05-15T16:03:34.862Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-alert-server: analyzer-1
2019-05-15T16:03:34.867Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-topology-manager: analyzer-1
2019-05-15T16:03:34.872Z	INFO	analyzer/analyzer.go:55 glob..func1	analyzer-1: Skydive Analyzer started !
2019-05-15T16:03:34.989Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:03:34.989Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: analyzer-1 < admin
2019-05-15T16:03:34.990Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/replication, analyzer-1
2019-05-15T16:03:34.990Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: analyzer-1, websocket, /ws/replication ---> true
2019-05-15T16:03:34.990Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client analyzer-1 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:03:34.990Z	INFO	websocket/client.go:519 (*Client).Connect	analyzer-1: Connected to wss://[::1]:8082/ws/replication
2019-05-15T16:03:34.990Z	DEBUG	hub/replication_endpoint.go:88 (*ReplicatorPeer).OnConnected	analyzer-1: Disconnecting from wss://[::1]:8082/ws/replication since it's me
2019-05-15T16:03:34.990Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from [::1]:48006 : URI path /ws/replication
2019-05-15T16:03:34.991Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient analyzer-1 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:03:34.991Z	ERROR	websocket/client.go:332 (*Conn).run.func3	analyzer-1: Error while writing to the WebSocket: write tcp [::1]:8082->[::1]:48006: write: broken pipe
2019-05-15T16:03:34.991Z	DEBUG	websocket/pool.go:85 (*Pool).OnDisconnected	analyzer-1: OnDisconnected analyzer-1 for pool /ws/replication type : [*websocket.Pool] 
2019-05-15T16:03:34.991Z	DEBUG	websocket/pool.go:143 (*Pool).RemoveClient	analyzer-1: Successfully removed client analyzer-1 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:03:35.247Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:03:35.247Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: analyzer-1 < admin, analyzer-2 < admin
2019-05-15T16:03:35.248Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/replication, analyzer-2
2019-05-15T16:03:35.248Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: analyzer-2, websocket, /ws/replication ---> true
2019-05-15T16:03:35.248Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client analyzer-2 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:03:35.249Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from [::1]:48032 : URI path /ws/replication
2019-05-15T16:03:35.249Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient analyzer-2 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:03:35.408Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-es-rolling-index:3f0e228a: analyzer-1
2019-05-15T16:03:35.408Z	INFO	elasticsearch/client.go:225 (*Client).start	analyzer-1: client started for skydive_flow, skydive_metric, skydive_rawpacket
2019-05-15T16:03:35.806Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:03:35.806Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: analyzer-2 < admin, agent-2 < admin, analyzer-1 < admin
2019-05-15T16:03:35.810Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/topology, agent-2
2019-05-15T16:03:35.810Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-2, websocket, /ws/agent/topology ---> true
2019-05-15T16:03:35.810Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-2 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:03:35.810Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.2:43492 : URI path /ws/agent/topology
2019-05-15T16:03:35.810Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-2 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:03:35.813Z	DEBUG	hub/pod_endpoint.go:70 (*AgentEndpoint).OnStructMessage	analyzer-1: Authoritative client registered agent.agent-2
2019-05-15T16:03:35.816Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:03:35.816Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: analyzer-1 < admin, analyzer-2 < admin, agent-2 < admin, agent-1 < admin
2019-05-15T16:03:35.817Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/topology, agent-1
2019-05-15T16:03:35.817Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-1, websocket, /ws/agent/topology ---> true
2019-05-15T16:03:35.817Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-1 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:03:35.817Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.1:37994 : URI path /ws/agent/topology
2019-05-15T16:03:35.817Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-1 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:03:35.837Z	INFO	websocket/client.go:495 (*Client).Connect	analyzer-1: Connecting to wss://[::1]:8084/ws/replication
2019-05-15T16:03:35.912Z	INFO	websocket/client.go:519 (*Client).Connect	analyzer-1: Connected to wss://[::1]:8084/ws/replication
2019-05-15T16:03:35.912Z	DEBUG	hub/replication_endpoint.go:95 (*ReplicatorPeer).OnConnected	analyzer-1: Disconnecting from wss://[::1]:8084/ws/replication as already connected through wss://[::1]:8084/ws/replication
2019-05-15T16:04:06.568Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:04:06.568Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: agent-2 < admin, agent-1 < admin, analyzer-1 < admin, admin < admin, analyzer-2 < admin
2019-05-15T16:04:06.569Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:04:06.664Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:04:06.668Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, capture, write ---> true
2019-05-15T16:04:06.671Z	DEBUG	client/client.go:357 (*OnDemandProbeClient).onAPIWatcherEvent	analyzer-1: New watcher event set for 92d22251-51e9-4f92-623a-e86435087fa9
2019-05-15T16:04:06.682Z	DEBUG	client/client.go:83 (*OnDemandProbeClient).OnStructMessage	analyzer-1: Capture start request succeeded Namespace OnDemand Type CaptureStartReply UUID 3b9f7136-e123-47ff-5698-942ec3152742 Status 200 Obj (347 bytes)
2019-05-15T16:04:06.685Z	DEBUG	client/client.go:83 (*OnDemandProbeClient).OnStructMessage	analyzer-1: Capture start request succeeded Namespace OnDemand Type CaptureStartReply UUID 7db94e75-a531-443a-7ef9-728df05189b5 Status 200 Obj (347 bytes)
2019-05-15T16:04:21.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:1f1d9ccaf7e668b8 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14279  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936257151 Last:1557936261678 RTT:160000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936257151 Last:1557936257151 RTT:160000  TCPMetric:<nil> IPMetric: Start:1557936257151 Last:1557936257151 TrackingID:1726323f08280d19 L3TrackingID:c2de13cdb6f0a6af ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:21.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:a6dcc04dd07631e6 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14314  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936259421 Last:1557936261678 RTT:132000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936259421 Last:1557936259421 RTT:132000  TCPMetric:<nil> IPMetric: Start:1557936259421 Last:1557936259421 TrackingID:a5306a7ffc3e4032 L3TrackingID:71379105dd8aa1ac ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:21.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9bd16b5b278477fe LayersPath:Ethernet/LLC/STP Application:STP Link:A:"6e:9b:e9:40:fd:9d" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936256678 Last:1557936261678  Metric:ABPackets:7 ABBytes:371 Start:1557936248381 Last:1557936260396  TCPMetric:<nil> IPMetric:<nil> Start:1557936248381 Last:1557936260396 TrackingID:74cc46c84cb1297a L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:21.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:aedcb166ef0c80d5 LayersPath:Ethernet/ARP Application:ARP Link:A:"66:91:2b:25:84:85" B:"ff:ff:ff:ff:ff:ff"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:Start:1557936251678 Last:1557936256678  Metric:ABPackets:1 ABBytes:42 Start:1557936246915 Last:1557936246915  TCPMetric:<nil> IPMetric:<nil> Start:1557936246915 Last:1557936246915 TrackingID:0e655dd5139e2eb3 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:TIMEOUT}
2019-05-15T16:04:21.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:4bbf3148d72c27e4 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14295  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936258277 Last:1557936261678 RTT:101000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936258277 Last:1557936258277 RTT:101000  TCPMetric:<nil> IPMetric: Start:1557936258277 Last:1557936258277 TrackingID:9454337f2ae3e677 L3TrackingID:8acb169a5cdaaad9 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:21.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:aed646a2aafc3a5d LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14330  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936260576 Last:1557936261678 RTT:95000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936260576 Last:1557936260576 RTT:95000  TCPMetric:<nil> IPMetric: Start:1557936260576 Last:1557936260576 TrackingID:d2ad84248fb5c691 L3TrackingID:5ff737428fd7d797 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:21.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:6bfcc885c8d73030 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14260  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936256023 Last:1557936261678 RTT:210000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936256023 Last:1557936256024 RTT:210000  TCPMetric:<nil> IPMetric: Start:1557936256023 Last:1557936256024 TrackingID:3ff9f2b7f3f5153d L3TrackingID:f1369a0f617c6906 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:24.873Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 7 flows stored
2019-05-15T16:04:26.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:0334b57cd3fda64c LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14362  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936262818 Last:1557936266678 RTT:94000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936262818 Last:1557936262818 RTT:94000  TCPMetric:<nil> IPMetric: Start:1557936262818 Last:1557936262818 TrackingID:a66f1781045d074a L3TrackingID:e9301c54588a9295 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:26.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:de796ea3cfbbaf00 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14396  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936265171 Last:1557936266678 RTT:240000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936265171 Last:1557936265171 RTT:240000  TCPMetric:<nil> IPMetric: Start:1557936265171 Last:1557936265171 TrackingID:10b503eacf0b81b5 L3TrackingID:aedcaef40ba0c6ef ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:26.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e2ce4b41bdabfc97 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14346  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936261691 Last:1557936266678 RTT:88000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936261691 Last:1557936261691 RTT:88000  TCPMetric:<nil> IPMetric: Start:1557936261691 Last:1557936261691 TrackingID:ab29fea89113d2d1 L3TrackingID:edde889c84bea6fe ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:26.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:7c9caa7b3e4373c4 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14380  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936264015 Last:1557936266678 RTT:108000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936264015 Last:1557936264015 RTT:108000  TCPMetric:<nil> IPMetric: Start:1557936264015 Last:1557936264015 TrackingID:8475963cb2b3cf47 L3TrackingID:97b0654af2e61012 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:26.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:390e1be60fb43675 LayersPath:Ethernet/ARP Application:ARP Link:A:"de:3d:74:1b:9b:d1" B:"66:91:2b:25:84:85"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:Start:1557936256678 Last:1557936261678  Metric:ABPackets:2 ABBytes:84 BAPackets:1 BABytes:42 Start:1557936246916 Last:1557936251967  TCPMetric:<nil> IPMetric:<nil> Start:1557936246916 Last:1557936251967 TrackingID:858437db2ac7adb9 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:TIMEOUT}
2019-05-15T16:04:26.680Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9bd16b5b278477fe LayersPath:Ethernet/LLC/STP Application:STP Link:A:"6e:9b:e9:40:fd:9d" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936261678 Last:1557936266678  Metric:ABPackets:9 ABBytes:477 Start:1557936248381 Last:1557936264401  TCPMetric:<nil> IPMetric:<nil> Start:1557936248381 Last:1557936264401 TrackingID:74cc46c84cb1297a L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:29.881Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 6 flows stored
2019-05-15T16:04:34.517Z	DEBUG	elasticsearch/rollindex.go:83 (*rollIndexService).roll	analyzer-1: Start rolling indices...
2019-05-15T16:04:34.517Z	INFO	elasticsearch/rollindex.go:120 (*rollIndexService).roll	analyzer-1: Rolling indices done
2019-05-15T16:04:35.409Z	DEBUG	elasticsearch/rollindex.go:83 (*rollIndexService).roll	analyzer-1: Start rolling indices...
2019-05-15T16:04:35.409Z	INFO	elasticsearch/rollindex.go:120 (*rollIndexService).roll	analyzer-1: Rolling indices done
2019-05-15T16:04:36.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9bd16b5b278477fe LayersPath:Ethernet/LLC/STP Application:STP Link:A:"6e:9b:e9:40:fd:9d" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936271678 Last:1557936276678  Metric:ABPackets:14 ABBytes:742 Start:1557936248381 Last:1557936274912  TCPMetric:<nil> IPMetric:<nil> Start:1557936248381 Last:1557936274912 TrackingID:74cc46c84cb1297a L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:36.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:802832590f828a50 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14519  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936273099 Last:1557936276678 RTT:120000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936273099 Last:1557936273099 RTT:120000  TCPMetric:<nil> IPMetric: Start:1557936273099 Last:1557936273099 TrackingID:d09004ca4b302cec L3TrackingID:24552e1be8120d86 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:36.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9032d1a0692a48f5 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14558  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936275350 Last:1557936276678 RTT:128000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936275350 Last:1557936275350 RTT:128000  TCPMetric:<nil> IPMetric: Start:1557936275350 Last:1557936275350 TrackingID:e8a439eb68002a69 L3TrackingID:4b3095972cc87c5a ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:36.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:603ddfb1fda2cb25 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"e6:f2:39:fc:c5:2e" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::e4f2:39ff:fefc:c52e" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936275007 Last:1557936276678  Metric:ABPackets:1 ABBytes:70 Start:1557936275007 Last:1557936275007  TCPMetric:<nil> IPMetric:<nil> Start:1557936275007 Last:1557936275007 TrackingID:9cb3d97f2c905612 L3TrackingID:e9a4b4698e19d898 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:36.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:6d5b57ff65998b48 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14538  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936274220 Last:1557936276678 RTT:99000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936274220 Last:1557936274220 RTT:99000  TCPMetric:<nil> IPMetric: Start:1557936274220 Last:1557936274220 TrackingID:737ed4dd03611e0b L3TrackingID:fdb7748a69b13402 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:36.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9521aa7650a080cc LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"de:3d:74:1b:9b:d1" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::dc3d:74ff:fe1b:9bd1" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936274495 Last:1557936276678  Metric:ABPackets:1 ABBytes:70 Start:1557936274495 Last:1557936274495  TCPMetric:<nil> IPMetric:<nil> Start:1557936274495 Last:1557936274495 TrackingID:bbf6654cdcdbbf29 L3TrackingID:0172e7491c22372b ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:36.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:04587238e55128a4 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14503  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936271973 Last:1557936276678 RTT:119000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936271973 Last:1557936271974 RTT:119000  TCPMetric:<nil> IPMetric: Start:1557936271973 Last:1557936271974 TrackingID:e3cfdd9c22e4f780 L3TrackingID:c0e69f1b42b7eac8 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:39.873Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 7 flows stored
2019-05-15T16:04:41.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:c58677ff709486d2 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14609  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936278708 Last:1557936281678 RTT:167000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936278708 Last:1557936278709 RTT:167000  TCPMetric:<nil> IPMetric: Start:1557936278708 Last:1557936278709 TrackingID:47fa97b408d304cb L3TrackingID:f10635009ae493f2 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:bdb99bb4ce82f3ce LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14626  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936279830 Last:1557936281678 RTT:140000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936279830 Last:1557936279830 RTT:140000  TCPMetric:<nil> IPMetric: Start:1557936279830 Last:1557936279830 TrackingID:788ec7ff3ee56a7b L3TrackingID:f2155c43d55621cd ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9bd16b5b278477fe LayersPath:Ethernet/LLC/STP Application:STP Link:A:"6e:9b:e9:40:fd:9d" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936276678 Last:1557936281678  Metric:ABPackets:17 ABBytes:901 Start:1557936248381 Last:1557936281422  TCPMetric:<nil> IPMetric:<nil> Start:1557936248381 Last:1557936281422 TrackingID:74cc46c84cb1297a L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:4da6a81716684e88 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"66:91:2b:25:84:85" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::6491:2bff:fe25:8485" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936278079 Last:1557936281678  Metric:ABPackets:1 ABBytes:70 Start:1557936278079 Last:1557936278079  TCPMetric:<nil> IPMetric:<nil> Start:1557936278079 Last:1557936278079 TrackingID:d68e79bc38938e38 L3TrackingID:eee8c47912f7fcda ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:bf73b848d40e6d6d LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14577  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936276464 Last:1557936281678 RTT:103000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936276464 Last:1557936276464 RTT:103000  TCPMetric:<nil> IPMetric: Start:1557936276464 Last:1557936276464 TrackingID:e3b9e71bde337dae L3TrackingID:ad309ad2f2bc9f7b ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:05623a19e5dfcec1 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14593  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936277584 Last:1557936281678 RTT:134000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936277584 Last:1557936277584 RTT:134000  TCPMetric:<nil> IPMetric: Start:1557936277584 Last:1557936277584 TrackingID:e03e54cf840e76b9 L3TrackingID:78ee4e54aa471f2d ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:659a0b0ecf34b872 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14593  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936277584 Last:1557936281683 RTT:25000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936277584 Last:1557936277584 RTT:25000  TCPMetric:<nil> IPMetric: Start:1557936277584 Last:1557936277584 TrackingID:e03e54cf840e76b9 L3TrackingID:78ee4e54aa471f2d ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:26b50455b8876eb2 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"fa:68:08:bf:36:62" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936276683 Last:1557936281683  Metric:ABPackets:17 ABBytes:901 Start:1557936248378 Last:1557936281422  TCPMetric:<nil> IPMetric:<nil> Start:1557936248378 Last:1557936281422 TrackingID:93f2975b00727ff4 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:4320c0c3cbcb8c41 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"66:91:2b:25:84:85" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::6491:2bff:fe25:8485" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936278079 Last:1557936281683  Metric:ABPackets:1 ABBytes:70 Start:1557936278079 Last:1557936278079  TCPMetric:<nil> IPMetric:<nil> Start:1557936278079 Last:1557936278079 TrackingID:d68e79bc38938e38 L3TrackingID:eee8c47912f7fcda ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:59249db688fd5da2 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14626  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936279830 Last:1557936281683 RTT:29000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936279830 Last:1557936279830 RTT:29000  TCPMetric:<nil> IPMetric: Start:1557936279830 Last:1557936279830 TrackingID:788ec7ff3ee56a7b L3TrackingID:f2155c43d55621cd ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:69d806473b627c11 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14577  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936276464 Last:1557936281683 RTT:20000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936276464 Last:1557936276464 RTT:20000  TCPMetric:<nil> IPMetric: Start:1557936276464 Last:1557936276464 TrackingID:e3b9e71bde337dae L3TrackingID:ad309ad2f2bc9f7b ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:41.685Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:279c05fa7bad414e LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"66:91:2b:25:84:85" B:"de:3d:74:1b:9b:d1"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:14609  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936278709 Last:1557936281683 RTT:37000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936278709 Last:1557936278709 RTT:37000  TCPMetric:<nil> IPMetric: Start:1557936278709 Last:1557936278709 TrackingID:47fa97b408d304cb L3TrackingID:f10635009ae493f2 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:44.873Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 12 flows stored
2019-05-15T16:04:46.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:477984e2c62f952f LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"f6:d0:7a:f0:69:cc" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::f4d0:7aff:fef0:69cc" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936282175 Last:1557936286678  Metric:ABPackets:1 ABBytes:70 Start:1557936282175 Last:1557936282175  TCPMetric:<nil> IPMetric:<nil> Start:1557936282175 Last:1557936282175 TrackingID:60291abc4ad9f816 L3TrackingID:0ae68fc2f6c6bc33 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:46.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9bd16b5b278477fe LayersPath:Ethernet/LLC/STP Application:STP Link:A:"6e:9b:e9:40:fd:9d" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936281678 Last:1557936286678  Metric:ABPackets:19 ABBytes:1007 Start:1557936248381 Last:1557936285427  TCPMetric:<nil> IPMetric:<nil> Start:1557936248381 Last:1557936285427 TrackingID:74cc46c84cb1297a L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:46.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:26b50455b8876eb2 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"fa:68:08:bf:36:62" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936281683 Last:1557936286683  Metric:ABPackets:19 ABBytes:1007 Start:1557936248378 Last:1557936285422  TCPMetric:<nil> IPMetric:<nil> Start:1557936248378 Last:1557936285422 TrackingID:93f2975b00727ff4 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:46.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:910c41b2a60d4efe LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"f6:d0:7a:f0:69:cc" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::f4d0:7aff:fef0:69cc" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936282175 Last:1557936286683  Metric:ABPackets:1 ABBytes:70 Start:1557936282175 Last:1557936282175  TCPMetric:<nil> IPMetric:<nil> Start:1557936282175 Last:1557936282175 TrackingID:60291abc4ad9f816 L3TrackingID:0ae68fc2f6c6bc33 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:49.873Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 4 flows stored
2019-05-15T16:04:52.848Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:04:52.848Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: analyzer-2 < admin, agent-2 < admin, agent-1 < admin, analyzer-1 < admin, agent-3 < admin, admin < admin
2019-05-15T16:04:52.852Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/topology, agent-3
2019-05-15T16:04:52.852Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-3, websocket, /ws/agent/topology ---> true
2019-05-15T16:04:52.852Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-3 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:04:52.857Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.3:58180 : URI path /ws/agent/topology
2019-05-15T16:04:52.858Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-3 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:04:52.858Z	DEBUG	hub/pod_endpoint.go:70 (*AgentEndpoint).OnStructMessage	analyzer-1: Authoritative client registered agent.agent-3
2019-05-15T16:04:53.890Z	DEBUG	client/client.go:83 (*OnDemandProbeClient).OnStructMessage	analyzer-1: Capture start request succeeded Namespace OnDemand Type CaptureStartReply UUID 434e0629-860a-4fe7-7257-a61ce9cba897 Status 200 Obj (347 bytes)
2019-05-15T16:04:55.034Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:04:55.034Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: agent-1 < admin, analyzer-1 < admin, admin < admin, agent-3 < admin, agent-4 < admin, analyzer-2 < admin, agent-2 < admin
2019-05-15T16:04:55.035Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/topology, agent-4
2019-05-15T16:04:55.036Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-4, websocket, /ws/agent/topology ---> true
2019-05-15T16:04:55.036Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-4 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:04:55.037Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.4:54500 : URI path /ws/agent/topology
2019-05-15T16:04:55.043Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-4 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:04:55.044Z	DEBUG	hub/pod_endpoint.go:70 (*AgentEndpoint).OnStructMessage	analyzer-1: Authoritative client registered agent.agent-4
2019-05-15T16:04:55.748Z	DEBUG	client/client.go:83 (*OnDemandProbeClient).OnStructMessage	analyzer-1: Capture start request succeeded Namespace OnDemand Type CaptureStartReply UUID 0575afc4-0fa2-4723-6c49-41107e9de40d Status 200 Obj (347 bytes)
2019-05-15T16:04:56.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9bd16b5b278477fe LayersPath:Ethernet/LLC/STP Application:STP Link:A:"6e:9b:e9:40:fd:9d" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936291679 Last:1557936296678  Metric:ABPackets:23 ABBytes:1219 Start:1557936248381 Last:1557936293604  TCPMetric:<nil> IPMetric:<nil> Start:1557936248381 Last:1557936293604 TrackingID:74cc46c84cb1297a L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:57.324Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:04:57.669Z	DEBUG	websocket/pool.go:85 (*Pool).OnDisconnected	analyzer-1: OnDisconnected agent-4 for pool /ws/agent/topology type : [*websocket.Pool] 
2019-05-15T16:04:57.669Z	DEBUG	hub/pod_endpoint.go:51 (*AgentEndpoint).OnDisconnected	analyzer-1: Authoritative client unregistered, delete resources of agent.agent-4
2019-05-15T16:04:57.684Z	DEBUG	websocket/pool.go:143 (*Pool).RemoveClient	analyzer-1: Successfully removed client agent-4 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:04:57.879Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:04:57.967Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, topology, read ---> true
2019-05-15T16:04:58.068Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:04:58.221Z	DEBUG	websocket/pool.go:85 (*Pool).OnDisconnected	analyzer-1: OnDisconnected analyzer-2 for pool /ws/replication type : [*websocket.Pool] 
2019-05-15T16:04:58.221Z	DEBUG	hub/replication_endpoint.go:387 (*ReplicationEndpoint).OnDisconnected	analyzer-1: Peer unregistered, delete resources of analyzer.analyzer-2
2019-05-15T16:04:58.222Z	DEBUG	websocket/pool.go:143 (*Pool).RemoveClient	analyzer-1: Successfully removed client analyzer-2 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:04:58.231Z	DEBUG	hub/pod_endpoint.go:70 (*AgentEndpoint).OnStructMessage	analyzer-1: Authoritative client registered agent.agent-1
2019-05-15T16:04:58.744Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:561df03b0d1b69ca LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"8a:80:58:0a:87:32" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::8880:58ff:fe0a:8732" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936294079 Last:1557936298744  Metric:ABPackets:1 ABBytes:70 Start:1557936294079 Last:1557936294079  TCPMetric:<nil> IPMetric:<nil> Start:1557936294079 Last:1557936294079 TrackingID:2645a9d9b3fc5d0c L3TrackingID:0ff0957dc1c38728 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:58.744Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:bf0559a51d4b1b8d LayersPath:Ethernet/LLC/STP Application:STP Link:A:"8a:80:58:0a:87:32" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:4 ABBytes:212 Start:1557936294098 Last:1557936298744  Metric:ABPackets:4 ABBytes:212 Start:1557936294098 Last:1557936297478  TCPMetric:<nil> IPMetric:<nil> Start:1557936294098 Last:1557936297478 TrackingID:1c82a912f7e36467 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:58.744Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:da15ef626bca46c2 LayersPath:Ethernet/IPv6/IPv6HopByHop/ICMPv6/MLDv2MulticastListenerReport Application:MLDv2MulticastListenerReport Link:A:"7a:d2:26:68:b8:66" B:"33:33:00:00:00:16"  Network:Protocol:IPV6 A:"fe80::78d2:26ff:fe68:b866" B:"ff02::16"  Transport:<nil> ICMP: DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:180 Start:1557936293760 Last:1557936298744  Metric:ABPackets:2 ABBytes:180 Start:1557936293760 Last:1557936294335  TCPMetric:<nil> IPMetric:<nil> Start:1557936293760 Last:1557936294335 TrackingID:f33628155f46e722 L3TrackingID:c6534dcc93e21c27 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:58.744Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:fd04a7f67ef94f9c LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"7a:d2:26:68:b8:66" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::78d2:26ff:fe68:b866" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:140 Start:1557936293760 Last:1557936298744  Metric:ABPackets:2 ABBytes:140 Start:1557936293760 Last:1557936297535  TCPMetric:<nil> IPMetric:<nil> Start:1557936293760 Last:1557936297535 TrackingID:39add8fd5a12dc60 L3TrackingID:146b29c1423f5753 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:58.744Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:8a70d7f522b6c001 LayersPath:Ethernet/IPv6/IPv6HopByHop/ICMPv6/MLDv2MulticastListenerReport Application:MLDv2MulticastListenerReport Link:A:"8a:80:58:0a:87:32" B:"33:33:00:00:00:16"  Network:Protocol:IPV6 A:"fe80::8880:58ff:fe0a:8732" B:"ff02::16"  Transport:<nil> ICMP: DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:180 Start:1557936294079 Last:1557936298744  Metric:ABPackets:2 ABBytes:180 Start:1557936294079 Last:1557936294359  TCPMetric:<nil> IPMetric:<nil> Start:1557936294079 Last:1557936294359 TrackingID:3a5271b4c59945da L3TrackingID:ce3ed5780a8e533d ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:04:59.873Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 6 flows stored
2019-05-15T16:05:01.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:d61062b7ccbbb864 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"b2:d0:da:44:f7:1f" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::b0d0:daff:fe44:f71f" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936299072 Last:1557936301678  Metric:ABPackets:1 ABBytes:70 Start:1557936299072 Last:1557936299072  TCPMetric:<nil> IPMetric:<nil> Start:1557936299072 Last:1557936299072 TrackingID:c5b3b5cbde27e8b2 L3TrackingID:f3a926248f3f64df ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:01.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9bd16b5b278477fe LayersPath:Ethernet/LLC/STP Application:STP Link:A:"6e:9b:e9:40:fd:9d" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936296678 Last:1557936301678  Metric:ABPackets:26 ABBytes:1378 Start:1557936248381 Last:1557936300205  TCPMetric:<nil> IPMetric:<nil> Start:1557936248381 Last:1557936300205 TrackingID:74cc46c84cb1297a L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:01.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:d2e71b04011f430a LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"7a:d2:26:68:b8:66" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::78d2:26ff:fe68:b866" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936297535 Last:1557936301678  Metric:ABPackets:1 ABBytes:70 Start:1557936297535 Last:1557936297535  TCPMetric:<nil> IPMetric:<nil> Start:1557936297535 Last:1557936297535 TrackingID:39add8fd5a12dc60 L3TrackingID:146b29c1423f5753 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:01.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e8207505b3372e22 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"7a:d2:26:68:b8:66" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::78d2:26ff:fe68:b866" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936297535 Last:1557936301683  Metric:ABPackets:1 ABBytes:70 Start:1557936297535 Last:1557936297535  TCPMetric:<nil> IPMetric:<nil> Start:1557936297535 Last:1557936297535 TrackingID:39add8fd5a12dc60 L3TrackingID:146b29c1423f5753 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:01.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:bab83417b408b5bf LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"b2:d0:da:44:f7:1f" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::b0d0:daff:fe44:f71f" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936299072 Last:1557936301683  Metric:ABPackets:1 ABBytes:70 Start:1557936299072 Last:1557936299072  TCPMetric:<nil> IPMetric:<nil> Start:1557936299072 Last:1557936299072 TrackingID:c5b3b5cbde27e8b2 L3TrackingID:f3a926248f3f64df ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:01.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:26b50455b8876eb2 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"fa:68:08:bf:36:62" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936296683 Last:1557936301683  Metric:ABPackets:26 ABBytes:1378 Start:1557936248378 Last:1557936299494  TCPMetric:<nil> IPMetric:<nil> Start:1557936248378 Last:1557936299494 TrackingID:93f2975b00727ff4 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:03.342Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:05:03.416Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, topology, read ---> true
2019-05-15T16:05:03.427Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:05:03.745Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:561df03b0d1b69ca LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"8a:80:58:0a:87:32" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::8880:58ff:fe0a:8732" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936298744 Last:1557936303744  Metric:ABPackets:2 ABBytes:140 Start:1557936294079 Last:1557936298559  TCPMetric:<nil> IPMetric:<nil> Start:1557936294079 Last:1557936298559 TrackingID:2645a9d9b3fc5d0c L3TrackingID:0ff0957dc1c38728 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:03.745Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:bf0559a51d4b1b8d LayersPath:Ethernet/LLC/STP Application:STP Link:A:"8a:80:58:0a:87:32" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936298744 Last:1557936303744  Metric:ABPackets:6 ABBytes:318 Start:1557936294098 Last:1557936301507  TCPMetric:<nil> IPMetric:<nil> Start:1557936294098 Last:1557936301507 TrackingID:1c82a912f7e36467 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:03.745Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:ec1861f855afafd9 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"b2:d0:da:44:f7:1f" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::b0d0:daff:fe44:f71f" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936299071 Last:1557936303744  Metric:ABPackets:1 ABBytes:70 Start:1557936299071 Last:1557936299071  TCPMetric:<nil> IPMetric:<nil> Start:1557936299071 Last:1557936299071 TrackingID:c5b3b5cbde27e8b2 L3TrackingID:f3a926248f3f64df ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:04.873Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 9 flows stored
2019-05-15T16:05:06.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:d2e71b04011f430a LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"7a:d2:26:68:b8:66" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::78d2:26ff:fe68:b866" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936301678 Last:1557936306678  Metric:ABPackets:2 ABBytes:140 Start:1557936297535 Last:1557936305215  TCPMetric:<nil> IPMetric:<nil> Start:1557936297535 Last:1557936305215 TrackingID:39add8fd5a12dc60 L3TrackingID:146b29c1423f5753 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:06.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:7a1beff476323dd1 LayersPath:Ethernet/ARP Application:ARP Link:A:"66:91:2b:25:84:85" B:"7a:d2:26:68:b8:66"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936303566 Last:1557936306678  Metric:ABPackets:1 ABBytes:42 Start:1557936303566 Last:1557936303566  TCPMetric:<nil> IPMetric:<nil> Start:1557936303566 Last:1557936303566 TrackingID:c378b97aee4c1407 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:06.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:be270086562d6377 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"7a:d2:26:68:b8:66" B:"66:91:2b:25:84:85"  Network:Protocol:IPV4 A:"10.0.3.1" B:"10.0.1.1"  Transport:<nil> ICMP:Type:ECHO ID:15126  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936303567 Last:1557936306678 RTT:74000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936303567 Last:1557936303567 RTT:74000  TCPMetric:<nil> IPMetric: Start:1557936303567 Last:1557936303567 TrackingID:e3d0f98876413920 L3TrackingID:3012f62ebe0a1015 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:06.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e1971c42091521b5 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"7a:d2:26:68:b8:66" B:"66:91:2b:25:84:85"  Network:Protocol:IPV4 A:"10.0.3.1" B:"10.0.1.1"  Transport:<nil> ICMP:Type:ECHO ID:15171  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936305866 Last:1557936306678 RTT:33000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936305866 Last:1557936305866 RTT:33000  TCPMetric:<nil> IPMetric: Start:1557936305866 Last:1557936305866 TrackingID:51da77cb8e980245 L3TrackingID:7aa5cdcf14aab4fb ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:06.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:dfe71f55c3d6ec9d LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"7a:d2:26:68:b8:66" B:"66:91:2b:25:84:85"  Network:Protocol:IPV4 A:"10.0.3.1" B:"10.0.1.1"  Transport:<nil> ICMP:Type:ECHO ID:15151  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936304722 Last:1557936306678 RTT:42000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936304722 Last:1557936304722 RTT:42000  TCPMetric:<nil> IPMetric: Start:1557936304722 Last:1557936304722 TrackingID:6b2c43aff420790d L3TrackingID:635867273b3f09d8 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:06.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:9bd16b5b278477fe LayersPath:Ethernet/LLC/STP Application:STP Link:A:"6e:9b:e9:40:fd:9d" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936301678 Last:1557936306678  Metric:ABPackets:28 ABBytes:1484 Start:1557936248381 Last:1557936304208  TCPMetric:<nil> IPMetric:<nil> Start:1557936248381 Last:1557936304208 TrackingID:74cc46c84cb1297a L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:06.679Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:34f607330d0efa15 LayersPath:Ethernet/ARP Application:ARP Link:A:"7a:d2:26:68:b8:66" B:"ff:ff:ff:ff:ff:ff"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936303566 Last:1557936306678  Metric:ABPackets:1 ABBytes:42 Start:1557936303566 Last:1557936303566  TCPMetric:<nil> IPMetric:<nil> Start:1557936303566 Last:1557936303566 TrackingID:287082dc9087f574 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:06.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:25b3ac004cb508a8 LayersPath:Ethernet/ARP Application:ARP Link:A:"7a:d2:26:68:b8:66" B:"ff:ff:ff:ff:ff:ff"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936303565 Last:1557936306683  Metric:ABPackets:1 ABBytes:42 Start:1557936303565 Last:1557936303565  TCPMetric:<nil> IPMetric:<nil> Start:1557936303565 Last:1557936303565 TrackingID:287082dc9087f574 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:06.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e8207505b3372e22 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"7a:d2:26:68:b8:66" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::78d2:26ff:fe68:b866" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936301683 Last:1557936306683  Metric:ABPackets:2 ABBytes:140 Start:1557936297535 Last:1557936305215  TCPMetric:<nil> IPMetric:<nil> Start:1557936297535 Last:1557936305215 TrackingID:39add8fd5a12dc60 L3TrackingID:146b29c1423f5753 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:06.684Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:26b50455b8876eb2 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"fa:68:08:bf:36:62" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936301683 Last:1557936306683  Metric:ABPackets:29 ABBytes:1537 Start:1557936248378 Last:1557936305785  TCPMetric:<nil> IPMetric:<nil> Start:1557936248378 Last:1557936305785 TrackingID:93f2975b00727ff4 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:08.744Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:fd04a7f67ef94f9c LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"7a:d2:26:68:b8:66" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::78d2:26ff:fe68:b866" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936303744 Last:1557936308744  Metric:ABPackets:3 ABBytes:210 Start:1557936293760 Last:1557936305215  TCPMetric:<nil> IPMetric:<nil> Start:1557936293760 Last:1557936305215 TrackingID:39add8fd5a12dc60 L3TrackingID:146b29c1423f5753 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:05:08.744Z	DEBUG	analyzer/flow_server.go:169 (*FlowServerUDPConn).Serve.func1	analyzer-1: New flow from UDP connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:561df03b0d1b69ca LayersPaif [ $status -ne 0 ] ; then
   echo "test Scale TLS:${TLS} FLOW_PROTOCOL:${FLOW_PROTOCOL} failed return ${status}"
   exit $status
fi

exit 0
[skydive-scale-tests] $ /bin/bash /tmp/jenkins5951213339297829562.sh
docker0
erspan0
eth0
gre0
gretap0
lo
lxdbr0
virbr0-nic
gobuild-cache
VOLUME
/usr/bin/minikube
Deleting local Kubernetes cluster...
Errors occurred deleting machine:  Error deleting host: minikube: Error loading host from store: Host does not exist: "minikube"
Total reclaimed space: 0B
which: no skydive in (/sbin:/usr/sbin:/usr/bin:/bin:/usr/local/sbin:/usr/local/bin)
readlink: missing operand
Try 'readlink --help' for more information.
+ check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop == start ']'
+ '[' stop == stop ']'
+ '[' -z 10 ']'
+ stop 10 10 10
+ ANALYZER_NUM=10
+ AGENT_NUM=10
+ VM_NUM=10
+ echo 'Stopping Agents...'
Stopping Agents...
++ seq 10
+ for A in $( seq $AGENT_NUM )
+ stop_agent 1
+ AGENT=1
+ NAME=agent-1
+ '[' -e /tmp/skydive-scale/agent-1.pid ']'
+ sudo pkill -f agent-1.yml
+ timeout=30
+ pkill -0 -f agent-1.yml
+ sudo rm /tmp/skydive-scale/agent-1.lock /tmp/skydive-scale/agent-1.pid
rm: cannot remove '/tmp/skydive-scale/agent-1.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-1.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 2
+ AGENT=2
+ NAME=agent-2
+ '[' -e /tmp/skydive-scale/agent-2.pid ']'
+ sudo pkill -f agent-2.yml
+ timeout=30
+ pkill -0 -f agent-2.yml
+ sudo rm /tmp/skydive-scale/agent-2.lock /tmp/skydive-scale/agent-2.pid
rm: cannot remove '/tmp/skydive-scale/agent-2.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-2.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 3
+ AGENT=3
+ NAME=agent-3
+ '[' -e /tmp/skydive-scale/agent-3.pid ']'
+ sudo pkill -f agent-3.yml
+ timeout=30
+ pkill -0 -f agent-3.yml
+ sudo rm /tmp/skydive-scale/agent-3.lock /tmp/skydive-scale/agent-3.pid
rm: cannot remove '/tmp/skydive-scale/agent-3.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-3.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 4
+ AGENT=4
+ NAME=agent-4
+ '[' -e /tmp/skydive-scale/agent-4.pid ']'
+ sudo pkill -f agent-4.yml
+ timeout=30
+ pkill -0 -f agent-4.yml
+ sudo rm /tmp/skydive-scale/agent-4.lock /tmp/skydive-scale/agent-4.pid
rm: cannot remove '/tmp/skydive-scale/agent-4.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-4.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 5
+ AGENT=5
+ NAME=agent-5
+ '[' -e /tmp/skydive-scale/agent-5.pid ']'
+ sudo pkill -f agent-5.yml
+ timeout=30
+ pkill -0 -f agent-5.yml
+ sudo rm /tmp/skydive-scale/agent-5.lock /tmp/skydive-scale/agent-5.pid
rm: cannot remove '/tmp/skydive-scale/agent-5.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-5.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 6
+ AGENT=6
+ NAME=agent-6
+ '[' -e /tmp/skydive-scale/agent-6.pid ']'
+ sudo pkill -f agent-6.yml
+ timeout=30
+ pkill -0 -f agent-6.yml
+ sudo rm /tmp/skydive-scale/agent-6.lock /tmp/skydive-scale/agent-6.pid
rm: cannot remove '/tmp/skydive-scale/agent-6.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-6.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 7
+ AGENT=7
+ NAME=agent-7
+ '[' -e /tmp/skydive-scale/agent-7.pid ']'
+ sudo pkill -f agent-7.yml
+ timeout=30
+ pkill -0 -f agent-7.yml
+ sudo rm /tmp/skydive-scale/agent-7.lock /tmp/skydive-scale/agent-7.pid
rm: cannot remove '/tmp/skydive-scale/agent-7.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-7.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 8
+ AGENT=8
+ NAME=agent-8
+ '[' -e /tmp/skydive-scale/agent-8.pid ']'
+ sudo pkill -f agent-8.yml
+ timeout=30
+ pkill -0 -f agent-8.yml
+ sudo rm /tmp/skydive-scale/agent-8.lock /tmp/skydive-scale/agent-8.pid
rm: cannot remove '/tmp/skydive-scale/agent-8.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-8.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 9
+ AGENT=9
+ NAME=agent-9
+ '[' -e /tmp/skydive-scale/agent-9.pid ']'
+ sudo pkill -f agent-9.yml
+ timeout=30
+ pkill -0 -f agent-9.yml
+ sudo rm /tmp/skydive-scale/agent-9.lock /tmp/skydive-scale/agent-9.pid
rm: cannot remove '/tmp/skydive-scale/agent-9.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-9.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 10
+ AGENT=10
+ NAME=agent-10
+ '[' -e /tmp/skydive-scale/agent-10.pid ']'
+ sudo pkill -f agent-10.yml
+ timeout=30
+ pkill -0 -f agent-10.yml
+ sudo rm /tmp/skydive-scale/agent-10.lock /tmp/skydive-scale/agent-10.pid
rm: cannot remove '/tmp/skydive-scale/agent-10.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-10.pid': No such file or directory
+ echo 'Stopping Analyzers...'
Stopping Analyzers...
++ seq 10
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 1
+ ANALYZER=1
+ NAME=analyzer-1
+ '[' -e /tmp/skydive-scale/analyzer-1.pid ']'
+ sudo pkill -f analyzer-1.yml
+ timeout=30
+ pkill -0 -f analyzer-1.yml
+ sudo rm /tmp/skydive-scale/analyzer-1.lock /tmp/skydive-scale/analyzer-1.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-1.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-1.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 2
+ ANALYZER=2
+ NAME=analyzer-2
+ '[' -e /tmp/skydive-scale/analyzer-2.pid ']'
+ sudo pkill -f analyzer-2.yml
+ timeout=30
+ pkill -0 -f analyzer-2.yml
+ sudo rm /tmp/skydive-scale/analyzer-2.lock /tmp/skydive-scale/analyzer-2.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-2.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-2.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 3
+ ANALYZER=3
+ NAME=analyzer-3
+ '[' -e /tmp/skydive-scale/analyzer-3.pid ']'
+ sudo pkill -f analyzer-3.yml
+ timeout=30
+ pkill -0 -f analyzer-3.yml
+ sudo rm /tmp/skydive-scale/analyzer-3.lock /tmp/skydive-scale/analyzer-3.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-3.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-3.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 4
+ ANALYZER=4
+ NAME=analyzer-4
+ '[' -e /tmp/skydive-scale/analyzer-4.pid ']'
+ sudo pkill -f analyzer-4.yml
+ timeout=30
+ pkill -0 -f analyzer-4.yml
+ sudo rm /tmp/skydive-scale/analyzer-4.lock /tmp/skydive-scale/analyzer-4.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-4.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-4.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 5
+ ANALYZER=5
+ NAME=analyzer-5
+ '[' -e /tmp/skydive-scale/analyzer-5.pid ']'
+ sudo pkill -f analyzer-5.yml
+ timeout=30
+ pkill -0 -f analyzer-5.yml
+ sudo rm /tmp/skydive-scale/analyzer-5.lock /tmp/skydive-scale/analyzer-5.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-5.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-5.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 6
+ ANALYZER=6
+ NAME=analyzer-6
+ '[' -e /tmp/skydive-scale/analyzer-6.pid ']'
+ sudo pkill -f analyzer-6.yml
+ timeout=30
+ pkill -0 -f analyzer-6.yml
+ sudo rm /tmp/skydive-scale/analyzer-6.lock /tmp/skydive-scale/analyzer-6.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-6.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-6.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 7
+ ANALYZER=7
+ NAME=analyzer-7
+ '[' -e /tmp/skydive-scale/analyzer-7.pid ']'
+ sudo pkill -f analyzer-7.yml
+ timeout=30
+ pkill -0 -f analyzer-7.yml
+ sudo rm /tmp/skydive-scale/analyzer-7.lock /tmp/skydive-scale/analyzer-7.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-7.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-7.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 8
+ ANALYZER=8
+ NAME=analyzer-8
+ '[' -e /tmp/skydive-scale/analyzer-8.pid ']'
+ sudo pkill -f analyzer-8.yml
+ timeout=30
+ pkill -0 -f analyzer-8.yml
+ sudo rm /tmp/skydive-scale/analyzer-8.lock /tmp/skydive-scale/analyzer-8.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-8.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-8.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 9
+ ANALYZER=9
+ NAME=analyzer-9
+ '[' -e /tmp/skydive-scale/analyzer-9.pid ']'
+ sudo pkill -f analyzer-9.yml
+ timeout=30
+ pkill -0 -f analyzer-9.yml
+ sudo rm /tmp/skydive-scale/analyzer-9.lock /tmp/skydive-scale/analyzer-9.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-9.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-9.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 10
+ ANALYZER=10
+ NAME=analyzer-10
+ '[' -e /tmp/skydive-scale/analyzer-10.pid ']'
+ sudo pkill -f analyzer-10.yml
+ timeout=30
+ pkill -0 -f analyzer-10.yml
+ sudo rm /tmp/skydive-scale/analyzer-10.lock /tmp/skydive-scale/analyzer-10.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-10.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/analyzer-10.pid': No such file or directory
+ sleep 5
+ sudo -E screen -S skydive-stress -X quit
No screen session found.
+ rm /tmp/skydive-scale/screen.lock
rm: cannot remove '/tmp/skydive-scale/screen.lock': No such file or directory
++ seq 10
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 1 10
+ IDX=1
+ VM_NUM=10
+ NAME=agent-1
+ echo Delete agent agent-1
Delete agent agent-1
++ cat /tmp/skydive-scale/agent-1-vswitchd.pid
cat: /tmp/skydive-scale/agent-1-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-1-dbserver.pid
cat: /tmp/skydive-scale/agent-1-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm1
Cannot remove namespace file "/var/run/netns/agent-1-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm2
Cannot remove namespace file "/var/run/netns/agent-1-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm3
Cannot remove namespace file "/var/run/netns/agent-1-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm4
Cannot remove namespace file "/var/run/netns/agent-1-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm5
Cannot remove namespace file "/var/run/netns/agent-1-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm6
Cannot remove namespace file "/var/run/netns/agent-1-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm7
Cannot remove namespace file "/var/run/netns/agent-1-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm8
Cannot remove namespace file "/var/run/netns/agent-1-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm9
Cannot remove namespace file "/var/run/netns/agent-1-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm10
Cannot remove namespace file "/var/run/netns/agent-1-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-1.lock
rm: cannot remove '/tmp/skydive-scale/agent-1.lock': No such file or directory
+ delete_host agent-1
+ NAME=agent-1
+ echo Delete Host agent-1
Delete Host agent-1
+ sudo ovs-vsctl del-port br-central agent-1-eth0
ovs-vsctl: no port named agent-1-eth0
+ sudo ip link del agent-1-eth0
Cannot find device "agent-1-eth0"
+ sudo ip netns del agent-1
Cannot remove namespace file "/var/run/netns/agent-1": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 2 10
+ IDX=2
+ VM_NUM=10
+ NAME=agent-2
+ echo Delete agent agent-2
Delete agent agent-2
++ cat /tmp/skydive-scale/agent-2-vswitchd.pid
cat: /tmp/skydive-scale/agent-2-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-2-dbserver.pid
cat: /tmp/skydive-scale/agent-2-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm1
Cannot remove namespace file "/var/run/netns/agent-2-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm2
Cannot remove namespace file "/var/run/netns/agent-2-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm3
Cannot remove namespace file "/var/run/netns/agent-2-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm4
Cannot remove namespace file "/var/run/netns/agent-2-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm5
Cannot remove namespace file "/var/run/netns/agent-2-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm6
Cannot remove namespace file "/var/run/netns/agent-2-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm7
Cannot remove namespace file "/var/run/netns/agent-2-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm8
Cannot remove namespace file "/var/run/netns/agent-2-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm9
Cannot remove namespace file "/var/run/netns/agent-2-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm10
Cannot remove namespace file "/var/run/netns/agent-2-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-2.lock
rm: cannot remove '/tmp/skydive-scale/agent-2.lock': No such file or directory
+ delete_host agent-2
+ NAME=agent-2
+ echo Delete Host agent-2
Delete Host agent-2
+ sudo ovs-vsctl del-port br-central agent-2-eth0
ovs-vsctl: no port named agent-2-eth0
+ sudo ip link del agent-2-eth0
Cannot find device "agent-2-eth0"
+ sudo ip netns del agent-2
Cannot remove namespace file "/var/run/netns/agent-2": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 3 10
+ IDX=3
+ VM_NUM=10
+ NAME=agent-3
+ echo Delete agent agent-3
Delete agent agent-3
++ cat /tmp/skydive-scale/agent-3-vswitchd.pid
cat: /tmp/skydive-scale/agent-3-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-3-dbserver.pid
cat: /tmp/skydive-scale/agent-3-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm1
Cannot remove namespace file "/var/run/netns/agent-3-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm2
Cannot remove namespace file "/var/run/netns/agent-3-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm3
Cannot remove namespace file "/var/run/netns/agent-3-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm4
Cannot remove namespace file "/var/run/netns/agent-3-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm5
Cannot remove namespace file "/var/run/netns/agent-3-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm6
Cannot remove namespace file "/var/run/netns/agent-3-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm7
Cannot remove namespace file "/var/run/netns/agent-3-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm8
Cannot remove namespace file "/var/run/netns/agent-3-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm9
Cannot remove namespace file "/var/run/netns/agent-3-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm10
Cannot remove namespace file "/var/run/netns/agent-3-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-3.lock
rm: cannot remove '/tmp/skydive-scale/agent-3.lock': No such file or directory
+ delete_host agent-3
+ NAME=agent-3
+ echo Delete Host agent-3
Delete Host agent-3
+ sudo ovs-vsctl del-port br-central agent-3-eth0
ovs-vsctl: no port named agent-3-eth0
+ sudo ip link del agent-3-eth0
Cannot find device "agent-3-eth0"
+ sudo ip netns del agent-3
Cannot remove namespace file "/var/run/netns/agent-3": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 4 10
+ IDX=4
+ VM_NUM=10
+ NAME=agent-4
+ echo Delete agent agent-4
Delete agent agent-4
++ cat /tmp/skydive-scale/agent-4-vswitchd.pid
cat: /tmp/skydive-scale/agent-4-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-4-dbserver.pid
cat: /tmp/skydive-scale/agent-4-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm1
Cannot remove namespace file "/var/run/netns/agent-4-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm2
Cannot remove namespace file "/var/run/netns/agent-4-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm3
Cannot remove namespace file "/var/run/netns/agent-4-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm4
Cannot remove namespace file "/var/run/netns/agent-4-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm5
Cannot remove namespace file "/var/run/netns/agent-4-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm6
Cannot remove namespace file "/var/run/netns/agent-4-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm7
Cannot remove namespace file "/var/run/netns/agent-4-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm8
Cannot remove namespace file "/var/run/netns/agent-4-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm9
Cannot remove namespace file "/var/run/netns/agent-4-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm10
Cannot remove namespace file "/var/run/netns/agent-4-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-4.lock
rm: cannot remove '/tmp/skydive-scale/agent-4.lock': No such file or directory
+ delete_host agent-4
+ NAME=agent-4
+ echo Delete Host agent-4
Delete Host agent-4
+ sudo ovs-vsctl del-port br-central agent-4-eth0
ovs-vsctl: no port named agent-4-eth0
+ sudo ip link del agent-4-eth0
Cannot find device "agent-4-eth0"
+ sudo ip netns del agent-4
Cannot remove namespace file "/var/run/netns/agent-4": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 5 10
+ IDX=5
+ VM_NUM=10
+ NAME=agent-5
+ echo Delete agent agent-5
Delete agent agent-5
++ cat /tmp/skydive-scale/agent-5-vswitchd.pid
cat: /tmp/skydive-scale/agent-5-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-5-dbserver.pid
cat: /tmp/skydive-scale/agent-5-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm1
Cannot remove namespace file "/var/run/netns/agent-5-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm2
Cannot remove namespace file "/var/run/netns/agent-5-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm3
Cannot remove namespace file "/var/run/netns/agent-5-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm4
Cannot remove namespace file "/var/run/netns/agent-5-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm5
Cannot remove namespace file "/var/run/netns/agent-5-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm6
Cannot remove namespace file "/var/run/netns/agent-5-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm7
Cannot remove namespace file "/var/run/netns/agent-5-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm8
Cannot remove namespace file "/var/run/netns/agent-5-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm9
Cannot remove namespace file "/var/run/netns/agent-5-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-5-vm10
Cannot remove namespace file "/var/run/netns/agent-5-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-5.lock
rm: cannot remove '/tmp/skydive-scale/agent-5.lock': No such file or directory
+ delete_host agent-5
+ NAME=agent-5
+ echo Delete Host agent-5
Delete Host agent-5
+ sudo ovs-vsctl del-port br-central agent-5-eth0
ovs-vsctl: no port named agent-5-eth0
+ sudo ip link del agent-5-eth0
Cannot find device "agent-5-eth0"
+ sudo ip netns del agent-5
Cannot remove namespace file "/var/run/netns/agent-5": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 6 10
+ IDX=6
+ VM_NUM=10
+ NAME=agent-6
+ echo Delete agent agent-6
Delete agent agent-6
++ cat /tmp/skydive-scale/agent-6-vswitchd.pid
cat: /tmp/skydive-scale/agent-6-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-6-dbserver.pid
cat: /tmp/skydive-scale/agent-6-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm1
Cannot remove namespace file "/var/run/netns/agent-6-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm2
Cannot remove namespace file "/var/run/netns/agent-6-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm3
Cannot remove namespace file "/var/run/netns/agent-6-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm4
Cannot remove namespace file "/var/run/netns/agent-6-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm5
Cannot remove namespace file "/var/run/netns/agent-6-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm6
Cannot remove namespace file "/var/run/netns/agent-6-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm7
Cannot remove namespace file "/var/run/netns/agent-6-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm8
Cannot remove namespace file "/var/run/netns/agent-6-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm9
Cannot remove namespace file "/var/run/netns/agent-6-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-6-vm10
Cannot remove namespace file "/var/run/netns/agent-6-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-6.lock
rm: cannot remove '/tmp/skydive-scale/agent-6.lock': No such file or directory
+ delete_host agent-6
+ NAME=agent-6
+ echo Delete Host agent-6
Delete Host agent-6
+ sudo ovs-vsctl del-port br-central agent-6-eth0
ovs-vsctl: no port named agent-6-eth0
+ sudo ip link del agent-6-eth0
Cannot find device "agent-6-eth0"
+ sudo ip netns del agent-6
Cannot remove namespace file "/var/run/netns/agent-6": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 7 10
+ IDX=7
+ VM_NUM=10
+ NAME=agent-7
+ echo Delete agent agent-7
Delete agent agent-7
++ cat /tmp/skydive-scale/agent-7-vswitchd.pid
cat: /tmp/skydive-scale/agent-7-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-7-dbserver.pid
cat: /tmp/skydive-scale/agent-7-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm1
Cannot remove namespace file "/var/run/netns/agent-7-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm2
Cannot remove namespace file "/var/run/netns/agent-7-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm3
Cannot remove namespace file "/var/run/netns/agent-7-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm4
Cannot remove namespace file "/var/run/netns/agent-7-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm5
Cannot remove namespace file "/var/run/netns/agent-7-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm6
Cannot remove namespace file "/var/run/netns/agent-7-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm7
Cannot remove namespace file "/var/run/netns/agent-7-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm8
Cannot remove namespace file "/var/run/netns/agent-7-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm9
Cannot remove namespace file "/var/run/netns/agent-7-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-7-vm10
Cannot remove namespace file "/var/run/netns/agent-7-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-7.lock
rm: cannot remove '/tmp/skydive-scale/agent-7.lock': No such file or directory
+ delete_host agent-7
+ NAME=agent-7
+ echo Delete Host agent-7
Delete Host agent-7
+ sudo ovs-vsctl del-port br-central agent-7-eth0
ovs-vsctl: no port named agent-7-eth0
+ sudo ip link del agent-7-eth0
Cannot find device "agent-7-eth0"
+ sudo ip netns del agent-7
Cannot remove namespace file "/var/run/netns/agent-7": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 8 10
+ IDX=8
+ VM_NUM=10
+ NAME=agent-8
+ echo Delete agent agent-8
Delete agent agent-8
++ cat /tmp/skydive-scale/agent-8-vswitchd.pid
cat: /tmp/skydive-scale/agent-8-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-8-dbserver.pid
cat: /tmp/skydive-scale/agent-8-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm1
Cannot remove namespace file "/var/run/netns/agent-8-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm2
Cannot remove namespace file "/var/run/netns/agent-8-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm3
Cannot remove namespace file "/var/run/netns/agent-8-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm4
Cannot remove namespace file "/var/run/netns/agent-8-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm5
Cannot remove namespace file "/var/run/netns/agent-8-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm6
Cannot remove namespace file "/var/run/netns/agent-8-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm7
Cannot remove namespace file "/var/run/netns/agent-8-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm8
Cannot remove namespace file "/var/run/netns/agent-8-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm9
Cannot remove namespace file "/var/run/netns/agent-8-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-8-vm10
Cannot remove namespace file "/var/run/netns/agent-8-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-8.lock
rm: cannot remove '/tmp/skydive-scale/agent-8.lock': No such file or directory
+ delete_host agent-8
+ NAME=agent-8
+ echo Delete Host agent-8
Delete Host agent-8
+ sudo ovs-vsctl del-port br-central agent-8-eth0
ovs-vsctl: no port named agent-8-eth0
+ sudo ip link del agent-8-eth0
Cannot find device "agent-8-eth0"
+ sudo ip netns del agent-8
Cannot remove namespace file "/var/run/netns/agent-8": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 9 10
+ IDX=9
+ VM_NUM=10
+ NAME=agent-9
+ echo Delete agent agent-9
Delete agent agent-9
++ cat /tmp/skydive-scale/agent-9-vswitchd.pid
cat: /tmp/skydive-scale/agent-9-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-9-dbserver.pid
cat: /tmp/skydive-scale/agent-9-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm1
Cannot remove namespace file "/var/run/netns/agent-9-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm2
Cannot remove namespace file "/var/run/netns/agent-9-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm3
Cannot remove namespace file "/var/run/netns/agent-9-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm4
Cannot remove namespace file "/var/run/netns/agent-9-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm5
Cannot remove namespace file "/var/run/netns/agent-9-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm6
Cannot remove namespace file "/var/run/netns/agent-9-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm7
Cannot remove namespace file "/var/run/netns/agent-9-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm8
Cannot remove namespace file "/var/run/netns/agent-9-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm9
Cannot remove namespace file "/var/run/netns/agent-9-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-9-vm10
Cannot remove namespace file "/var/run/netns/agent-9-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-9.lock
rm: cannot remove '/tmp/skydive-scale/agent-9.lock': No such file or directory
+ delete_host agent-9
+ NAME=agent-9
+ echo Delete Host agent-9
Delete Host agent-9
+ sudo ovs-vsctl del-port br-central agent-9-eth0
ovs-vsctl: no port named agent-9-eth0
+ sudo ip link del agent-9-eth0
Cannot find device "agent-9-eth0"
+ sudo ip netns del agent-9
Cannot remove namespace file "/var/run/netns/agent-9": No such file or directory
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 10 10
+ IDX=10
+ VM_NUM=10
+ NAME=agent-10
+ echo Delete agent agent-10
Delete agent agent-10
++ cat /tmp/skydive-scale/agent-10-vswitchd.pid
cat: /tmp/skydive-scale/agent-10-vswitchd.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ cat /tmp/skydive-scale/agent-10-dbserver.pid
cat: /tmp/skydive-scale/agent-10-dbserver.pid: No such file or directory
+ sudo kill -9
kill: not enough arguments
++ seq 10
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm1
Cannot remove namespace file "/var/run/netns/agent-10-vm1": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm2
Cannot remove namespace file "/var/run/netns/agent-10-vm2": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm3
Cannot remove namespace file "/var/run/netns/agent-10-vm3": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm4
Cannot remove namespace file "/var/run/netns/agent-10-vm4": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm5
Cannot remove namespace file "/var/run/netns/agent-10-vm5": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm6
Cannot remove namespace file "/var/run/netns/agent-10-vm6": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm7
Cannot remove namespace file "/var/run/netns/agent-10-vm7": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm8
Cannot remove namespace file "/var/run/netns/agent-10-vm8": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm9
Cannot remove namespace file "/var/run/netns/agent-10-vm9": No such file or directory
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-10-vm10
Cannot remove namespace file "/var/run/netns/agent-10-vm10": No such file or directory
+ rm /tmp/skydive-scale/agent-10.lock
rm: cannot remove '/tmp/skydive-scale/agent-10.lock': No such file or directory
+ delete_host agent-10
+ NAME=agent-10
+ echo Delete Host agent-10
Delete Host agent-10
+ sudo ovs-vsctl del-port br-central agent-10-eth0
ovs-vsctl: no port named agent-10-eth0
+ sudo ip link del agent-10-eth0
Cannot find device "agent-10-eth0"
+ sudo ip netns del agent-10
Cannot remove namespace file "/var/run/netns/agent-10": No such file or directory
+ delete_main_bridge
+ echo Delete central bridge
Delete central bridge
+ sudo ip link del br-central
Cannot find device "br-central"
+ sudo find /tmp/skydive-scale -mindepth 1 '!' -name '*.log' -a '!' -name '*.cover' -exec rm -rf '{}' ';'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100    21  100    21    0     0     95      0 --:--:-- --:--:-- --:--:--    95
{"acknowledged":true}
OrientDB console v.2.2.17 (build 2.2.x@rd9bace82ea8437117fd48114fc255e791056014b; 2017-02-16 17:20:27+0000) www.orientdb.com
Type 'help' to display all the supported commands.
Installing extensions for GREMLIN language v.2.6.0

orientdb> DROP DATABASE remote:localhost/Skydive root root plocal

Error: com.orientechnologies.orient.core.exception.OStorageException: Cannot delete the remote storage: Skydive

Error: com.orientechnologies.orient.core.exception.OStorageException: Database with name 'Skydive' does not exist
	DB name="Skydive"
	DB name="Skydive"

error: failed to get domain 'dev_dev'
error: Domain not found: no domain with matching name 'dev_dev'

error: failed to get domain 'dev_dev'
error: Domain not found: no domain with matching name 'dev_dev'

error: failed to get vol 'dev_dev.img'
error: Storage volume not found: no storage vol with matching path 'dev_dev.img'

error: failed to get domain 'vagrant_analyzer1'
error: Domain not found: no domain with matching name 'vagrant_analyzer1'

error: failed to get domain 'vagrant_analyzer1'
error: Domain not found: no domain with matching name 'vagrant_analyzer1'

error: failed to get vol 'vagrant_analyzer1.img'
error: Storage volume not found: no storage vol with matching path 'vagrant_analyzer1.img'

error: failed to get domain 'vagrant_agent1'
error: Domain not found: no domain with matching name 'vagrant_agent1'

error: failed to get domain 'vagrant_agent1'
error: Domain not found: no domain with matching name 'vagrant_agent1'

error: failed to get vol 'vagrant_agent1.img'
error: Storage volume not found: no storage vol with matching path 'vagrant_agent1.img'

error: failed to get domain 'devstack_devstack'
error: Domain not found: no domain with matching name 'devstack_devstack'

error: failed to get domain 'devstack_devstack'
error: Domain not found: no domain with matching name 'devstack_devstack'

error: failed to get vol 'devstack_devstack.img'
error: Storage volume not found: no storage vol with matching path 'devstack_devstack.img'

error: failed to get network 'vagrant0'
error: Network not found: no network with matching name 'vagrant0'

error: Failed to destroy network vagrant-libvirt
error: Requested operation is not valid: network 'vagrant-libvirt' is not active

[skydive-scale-tests] $ /bin/bash /tmp/jenkins9114762244413040172.sh

# Set Environment
echo ${PATH} | grep -q "${HOME}/bin" || {
  echo "Adding ${HOME}/bin to PATH"
  export PATH="${PATH}:${HOME}/bin"
}
Adding /var/lib/jenkins/bin to PATH

# Install Go
mkdir -p ~/bin
curl -sL -o ~/bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/master/gimme
chmod +x ~/bin/gimme

# before changing this be sure that it will not break the RHEL packaging
eval "$(gimme 1.10.3)"

unset GOOS;
unset GOARCH;
export GOROOT='/var/lib/jenkins/.gimme/versions/go1.10.3.linux.amd64';
export PATH="/var/lib/jenkins/.gimme/versions/go1.10.3.linux.amd64/bin:${PATH}";
go version >&2;
go version go1.10.3 linux/amd64

export GIMME_ENV='/var/lib/jenkins/.gimme/envs/go1.10.3.linux.amd64.env';

export GOPATH=$WORKSPACE
export PATH=$PATH:$GOPATH/bin

# speedup govendor sync command
mkdir -p $HOME/.govendor $GOPATH/.cache
rm -rf $GOPATH/.cache/govendor
ln -s $HOME/.govendor $GOPATH/.cache/govendor

# share compile cache
mkdir -p $HOME/pkg
rm -rf $GOPATH/pkg
ln -s $HOME/pkg $GOPATH/pkg
[[ "$ghprbCommentBody" = *"insanelock"* ]] && export TAGS=mutexdebug
FLOW_PROTOCOL=websocket scripts/ci/run-scale.sh

dir="$(dirname "$0")"

sudo iptables -F
sudo iptables -P FORWARD ACCEPT
for i in $(find /proc/sys/net/bridge/ -type f) ; do echo 0 | sudo tee $i ; done
0
0
0
0
0
0

cd ${GOPATH}/src/github.com/skydive-project/skydive
make install
CC= GOARCH= go get github.com/kardianos/govendor
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor sync
patch -p0 < dpdk/dpdk.govendor.patch
patching file vendor/github.com/intel-go/yanff/low/low.go
rm -rf vendor/github.com/weaveworks/tcptracer-bpf/vendor/github.com/
find vendor/github.com/docker/go-connections -name "*.go" | xargs -n 1 perl -i -pe 's|github.com/Sirupsen|github.com/sirupsen|g'
cd js && npm install
audited 930 packages in 2.045s
found 0 vulnerabilities

cd js && PATH=`npm bin`:$PATH make all
make[1]: Entering directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/js'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/js'
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/jteeuwen/go-bindata/go-bindata; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/jteeuwen/go-bindata/go-bindata); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; go-bindata  -nometadata -o statics/bindata.go -pkg=statics -ignore=bindata.go js/*.js rbac/policy.csv statics/index.html statics/css/* statics/css/themes/*/* statics/fonts/* statics/img/* statics/js/* statics/schemas/* statics/workflows/*.yaml 
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
gofmt -w -s statics/bindata.go
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/git.fd.io/govpp.git/cmd/binapi-generator; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install git.fd.io/govpp.git/cmd/binapi-generator); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor;
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor generate -tags " opencontrail lxd libvirt" \
	github.com/skydive-project/skydive/topology/probes/vpp
CGO_CFLAGS_ALLOW='.*' CGO_LDFLAGS_ALLOW='.*' /var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor install \
	-ldflags=" -B 0x871da3950c3509213f16601ed2d337786867afd8 -X github.com/skydive-project/skydive/version.Version=0.22.0-5bebd28d8830" \
	 -tags=" opencontrail lxd libvirt" -v \
	github.com/skydive-project/skydive

export SKYDIVE_ANALYZERS=localhost:8082
export ELASTICSEARCH=localhost:9200
export TLS=true
export SKYDIVE=${GOPATH}/bin/skydive
export FLOW_PROTOCOL=${FLOW_PROTOCOL:-websocket}
export SKYDIVE_LOGGING_LEVEL=DEBUG

make test.functionals WITH_SCALE=true TAGS=${TAGS} VERBOSE=true TIMEOUT=10m TEST_PATTERN=Scale
CC= GOARCH= go get github.com/kardianos/govendor
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor sync
patch -p0 < dpdk/dpdk.govendor.patch
patching file vendor/github.com/intel-go/yanff/low/low.go
rm -rf vendor/github.com/weaveworks/tcptracer-bpf/vendor/github.com/
find vendor/github.com/docker/go-connections -name "*.go" | xargs -n 1 perl -i -pe 's|github.com/Sirupsen|github.com/sirupsen|g'
cd js && npm install
up to date in 11.679s
cd js && PATH=`npm bin`:$PATH make all
make[1]: Entering directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/js'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/js'
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/github.com/jteeuwen/go-bindata/go-bindata; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install github.com/jteeuwen/go-bindata/go-bindata); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor; go-bindata  -nometadata -o statics/bindata.go -pkg=statics -ignore=bindata.go js/*.js rbac/policy.csv statics/index.html statics/css/* statics/css/themes/*/* statics/fonts/* statics/img/* statics/js/* statics/schemas/* statics/workflows/*.yaml 
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
gofmt -w -s statics/bindata.go
mv /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/git.fd.io/govpp.git/cmd/binapi-generator; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install git.fd.io/govpp.git/cmd/binapi-generator); cd -; mv bin /var/lib/jenkins/workspace/skydive-scale-tests/.skydive-build-tool; unlink src || mv src vendor;
/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor generate -tags " scale opencontrail lxd libvirt" \
	github.com/skydive-project/skydive/topology/probes/vpp
/var/lib/jenkins/workspace/skydive-scale-tests/bin/govendor test -tags " scale opencontrail lxd libvirt test"  -v -timeout 10m -c -o tests/functionals ./tests/
for functest in TestScaleHA ; do \
	make ARGS="-test.run $functest$\$ " test.functionals.run EXTRA_ARGS="-analyzer.topology.probes= -agent.topology.probes=opencontrail"; \
done
make[1]: Entering directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive'
cd tests && sudo -E ./functionals -test.v -test.timeout 10m -test.run TestScaleHA$  -analyzer.topology.probes= -agent.topology.probes=opencontrail
Config: ---
http:
  ws:
    pong_timeout: 10

analyzers:
  - 127.0.0.1:64500

analyzer:
  listen: 0.0.0.0:64500
  flow:
    backend: 
  analyzer_username: admin
  analyzer_password: password
  topology:
    backend: memory
    probes: 

  startup:
    capture_gremlin: "g.V().Has('Name','startup-vm2')"

agent:
  listen: 0.0.0.0:64499
  topology:
    probes:
    - ovsdb
    - docker
    - lxd
    - lldp
    - runc
    - socketinfo
    - libvirt
    - opencontrail

    netlink:
      metrics_update: 5
    lldp:
      interfaces:
      - lldp0

  metadata:
    mydict:
      value: 123
    myarrays:
      integers:
      - 1
      - 2
      - 3
      bools:
      - true
      - true
      strings:
      - dog
      - cat
      - frog

flow:
  expire: 600
  update: 10

ovs:
  oflow:
    enable: true
    native: false
    address:
      br-test: tcp://127.0.0.1:16633

storage:
  orientdb:
    addr: http://127.0.0.1:2480
    database: Skydive
    username: root
    password: 

logging:
  level: DEBUG

etcd:
  data_dir: /tmp/skydive-etcd
  embedded: true
  servers:
    - http://localhost:12379

=== RUN   TestScaleHA
2019-05-15T16:08:41.339Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh start 2 2 2]
2019-05-15T16:08:48.175Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' start == start ']'
+ '[' -z 2 ']'
+ start 2 2 2
+ ANALYZER_NUM=2
+ AGENT_NUM=2
+ VM_NUM=2
+ '[' '!' -e /tmp/skydive-scale/screen.lock ']'
+ sudo screen -dmS skydive-stress
+ sudo screen -ls
There is a screen on:
	19495.skydive-stress	(Detached)
1 Socket in /run/screen/S-root.
+ touch /tmp/skydive-scale/screen.lock
+ '[' true = true ']'
+ generate_ssl_conf 2 2
+ ANALYZER_NUM=2
+ AGENT_NUM=2
+ cat
++ seq 2
+ for I in $( seq $ANALYZER_NUM )
+ IDX=4
+ echo IP.4 = 192.168.50.1
+ for I in $( seq $ANALYZER_NUM )
+ IDX=5
+ echo IP.5 = 192.168.50.2
+ TOTAL_AGENT=7
++ seq 7
+ for I in $( seq $TOTAL_AGENT )
+ IP=3
+ IDX=5
+ echo IP.5 = 192.168.50.3
+ for I in $( seq $TOTAL_AGENT )
+ IP=4
+ IDX=6
+ echo IP.6 = 192.168.50.4
+ for I in $( seq $TOTAL_AGENT )
+ IP=5
+ IDX=7
+ echo IP.7 = 192.168.50.5
+ for I in $( seq $TOTAL_AGENT )
+ IP=6
+ IDX=8
+ echo IP.8 = 192.168.50.6
+ for I in $( seq $TOTAL_AGENT )
+ IP=7
+ IDX=9
+ echo IP.9 = 192.168.50.7
+ for I in $( seq $TOTAL_AGENT )
+ IP=8
+ IDX=10
+ echo IP.10 = 192.168.50.8
+ for I in $( seq $TOTAL_AGENT )
+ IP=9
+ IDX=11
+ echo IP.11 = 192.168.50.9
+ generate_tls_crt analyzer
+ NAME=analyzer
+ '[' -e /tmp/skydive-scale/analyzer.crt ']'
+ '[' '!' -e /tmp/skydive-scale/rootCA.crt ']'
+ sudo openssl genrsa -out /tmp/skydive-scale/rootCA.key 4096
Generating RSA private key, 4096 bit long modulus (2 primes)
................................++++
...................................................................++++
e is 65537 (0x010001)
+ sudo chmod 400 /tmp/skydive-scale/rootCA.key
+ yes ''
+ sudo openssl req -x509 -new -nodes -key /tmp/skydive-scale/rootCA.key -days 365 -out /tmp/skydive-scale/rootCA.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:State or Province Name (full name) []:Locality Name (eg, city) [Default City]:Organization Name (eg, company) [Default Company Ltd]:Organizational Unit Name (eg, section) []:Common Name (eg, your name or your server's hostname) []:Email Address []:+ sudo chmod 444 /tmp/skydive-scale/rootCA.crt
+ sudo openssl genrsa -out /tmp/skydive-scale/analyzer.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
....+++++
.....+++++
e is 65537 (0x010001)
+ sudo chmod 400 /tmp/skydive-scale/analyzer.key
+ yes ''
+ sudo openssl req -new -key /tmp/skydive-scale/analyzer.key -out /tmp/skydive-scale/analyzer.csr -subj /CN=analyzer -config /tmp/skydive-scale/skydive-ssl.cnf
+ sudo openssl x509 -req -days 365 -in /tmp/skydive-scale/analyzer.csr -CA /tmp/skydive-scale/rootCA.crt -CAkey /tmp/skydive-scale/rootCA.key -CAcreateserial -out /tmp/skydive-scale/analyzer.crt -extfile /tmp/skydive-scale/skydive-ssl.cnf -extensions v3_req
Signature ok
subject=CN = analyzer
Getting CA Private Key
+ sudo chmod 444 /tmp/skydive-scale/analyzer.crt
+ generate_tls_crt agent
+ NAME=agent
+ '[' -e /tmp/skydive-scale/agent.crt ']'
+ '[' '!' -e /tmp/skydive-scale/rootCA.crt ']'
+ sudo openssl genrsa -out /tmp/skydive-scale/agent.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
............................................................+++++
....+++++
e is 65537 (0x010001)
+ sudo chmod 400 /tmp/skydive-scale/agent.key
+ yes ''
+ sudo openssl req -new -key /tmp/skydive-scale/agent.key -out /tmp/skydive-scale/agent.csr -subj /CN=agent -config /tmp/skydive-scale/skydive-ssl.cnf
+ sudo openssl x509 -req -days 365 -in /tmp/skydive-scale/agent.csr -CA /tmp/skydive-scale/rootCA.crt -CAkey /tmp/skydive-scale/rootCA.key -CAcreateserial -out /tmp/skydive-scale/agent.crt -extfile /tmp/skydive-scale/skydive-ssl.cnf -extensions v3_req
Signature ok
subject=CN = agent
Getting CA Private Key
+ sudo chmod 444 /tmp/skydive-scale/agent.crt
+ create_main_bridge
+ echo Create central bridge
Create central bridge
+ sudo brctl addbr br-central
+ sudo brctl stp br-central off
+ sudo ip l set br-central up
+ sudo ip a add 192.168.50.254/24 dev br-central
+ start_inotify
+ '[' -f /tmp/skydive-scale/inotify.lock ']'
+ echo Start inotify netns watcher
Start inotify netns watcher
+ sudo mkdir -p /var/run/netns
+ cat
+ sudo -E screen -S skydive-stress -X screen -t skydive-inotify 0 bash -x /tmp/skydive-scale/inotify.sh
+ touch /tmp/skydive-scale/inotify.lock
+ WINDOW=1
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 1 -gt 1 ']'
+ '[' '!' -f /tmp/skydive-scale/analyzer-1.lock ']'
+ create_analyzer 1 2 2
+ IDX=1
+ ANALYZER_NUM=2
+ AGENT_NUM=2
+ NAME=analyzer-1
+ echo Create analyzer analyzer-1
Create analyzer analyzer-1
+ mkdir -p /tmp/skydive-scale/analyzer-1-etcd
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - localhost:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - localhost:8084'
+ STORAGE=
+ GRAPH=memory
+ '[' xlocalhost:9200 '!=' x ']'
+ STORAGE=elasticsearch
+ GRAPH=elasticsearch
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ cat
+ TOTAL_AGENT=7
++ seq 7
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_1'
+ echo '      - TOR_PORT_1 --> *[Name=agent-1]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_2'
+ echo '      - TOR_PORT_2 --> *[Name=agent-2]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_3'
+ echo '      - TOR_PORT_3 --> *[Name=agent-3]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_4'
+ echo '      - TOR_PORT_4 --> *[Name=agent-4]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_5'
+ echo '      - TOR_PORT_5 --> *[Name=agent-5]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_6'
+ echo '      - TOR_PORT_6 --> *[Name=agent-6]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_7'
+ echo '      - TOR_PORT_7 --> *[Name=agent-7]/eth0'
+ cat
++ seq 7
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-1: agent-password-1'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-2: agent-password-2'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-3: agent-password-3'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-4: agent-password-4'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-5: agent-password-5'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-6: agent-password-6'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-7: agent-password-7'
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-1: analyzer-password-1'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-2: analyzer-password-2'
+ COVERFILE=/tmp/skydive-scale/analyzer-1.cover
+ PIDFILE=/tmp/skydive-scale/analyzer-1.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-analyzer 1 sh -c 'export COVERFILE=/tmp/skydive-scale/analyzer-1.cover ; export PIDFILE=/tmp/skydive-scale/analyzer-1.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive analyzer -c /tmp/skydive-scale/analyzer-1.yml 2>&1 | tee /tmp/skydive-scale/analyzer-1.log'
+ touch /tmp/skydive-scale/analyzer-1.lock
+ CURR_ANALYZER_PORT=8084
+ WINDOW=2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 2 -gt 1 ']'
+ ETCD_EMBEDDED=false
+ '[' '!' -f /tmp/skydive-scale/analyzer-2.lock ']'
+ create_analyzer 2 2 2
+ IDX=2
+ ANALYZER_NUM=2
+ AGENT_NUM=2
+ NAME=analyzer-2
+ echo Create analyzer analyzer-2
Create analyzer analyzer-2
+ mkdir -p /tmp/skydive-scale/analyzer-2-etcd
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - localhost:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - localhost:8084'
+ STORAGE=
+ GRAPH=memory
+ '[' xlocalhost:9200 '!=' x ']'
+ STORAGE=elasticsearch
+ GRAPH=elasticsearch
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ cat
+ TOTAL_AGENT=7
++ seq 7
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_1'
+ echo '      - TOR_PORT_1 --> *[Name=agent-1]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_2'
+ echo '      - TOR_PORT_2 --> *[Name=agent-2]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_3'
+ echo '      - TOR_PORT_3 --> *[Name=agent-3]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_4'
+ echo '      - TOR_PORT_4 --> *[Name=agent-4]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_5'
+ echo '      - TOR_PORT_5 --> *[Name=agent-5]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_6'
+ echo '      - TOR_PORT_6 --> *[Name=agent-6]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_7'
+ echo '      - TOR_PORT_7 --> *[Name=agent-7]/eth0'
+ cat
++ seq 7
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-1: agent-password-1'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-2: agent-password-2'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-3: agent-password-3'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-4: agent-password-4'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-5: agent-password-5'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-6: agent-password-6'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-7: agent-password-7'
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-1: analyzer-password-1'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-2: analyzer-password-2'
+ COVERFILE=/tmp/skydive-scale/analyzer-2.cover
+ PIDFILE=/tmp/skydive-scale/analyzer-2.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-analyzer 2 sh -c 'export COVERFILE=/tmp/skydive-scale/analyzer-2.cover ; export PIDFILE=/tmp/skydive-scale/analyzer-2.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive analyzer -c /tmp/skydive-scale/analyzer-2.yml 2>&1 | tee /tmp/skydive-scale/analyzer-2.log'
+ touch /tmp/skydive-scale/analyzer-2.lock
+ CURR_ANALYZER_PORT=8086
+ WINDOW=3
+ AGENTS=
++ seq 2
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-1.lock ']'
+ create_host agent-1 192.168.50.1
+ NAME=agent-1
+ ADDR=192.168.50.1
+ echo Create host agent-1
Create host agent-1
+ sudo ip netns add agent-1
+ sudo ip netns exec agent-1 ip link set lo up
+ sudo ip link add agent-1-eth0 type veth peer name eth0 netns agent-1
+ sudo ip link set agent-1-eth0 up
+ sudo ip netns exec agent-1 ip link set eth0 up
+ sudo ip netns exec agent-1 ip address add 192.168.50.1/24 dev eth0
+ sudo ip netns exec agent-1 ip route add 0.0.0.0/0 via 192.168.50.254
+ sudo ip netns exec agent-1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-1-eth0
+ create_agent 1 192.168.50.1 2 2
+ IDX=1
+ ADDR=192.168.50.1
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-1
+ echo Create agent agent-1
Create agent agent-1
+ sudo ip netns exec agent-1 ovsdb-tool create /tmp/skydive-scale/agent-1.db
+ sudo ip netns exec agent-1 ovsdb-server /tmp/skydive-scale/agent-1.db --remote=punix:/tmp/skydive-scale/agent-1.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-1-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-1-dbserver.pid --detach
2019-05-15T16:08:43Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-1-vswitchd.log
+ sudo ip netns exec agent-1 ovs-vswitchd unix:/tmp/skydive-scale/agent-1.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-1-vswitchd.log --pidfile=/tmp/skydive-scale/agent-1-vswitchd.pid --detach --monitor
+ mkdir -p /tmp/skydive-scale/agent-1-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-1.cover
+ PIDFILE=/tmp/skydive-scale/agent-1.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 3 ip netns exec agent-1 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-1.cover ; export PIDFILE=/tmp/skydive-scale/agent-1.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-1.yml 2>&1 | tee /tmp/skydive-scale/agent-1.log'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-br agent-1
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock set bridge agent-1 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-1-vm1
Create VM agent-1-vm1
+ sudo ip netns add agent-1-vm1
+ sudo ip netns exec agent-1-vm1 ip l set lo up
+ sudo ip netns exec agent-1 ip link add vm1-eth0 type veth peer name eth0 netns agent-1-vm1
+ sudo ip netns exec agent-1 ip link set vm1-eth0 up
+ sudo ip netns exec agent-1-vm1 ip link set eth0 up
+ sudo ip netns exec agent-1-vm1 ip address add 10.0.1.1/16 dev eth0
+ sudo ip netns exec agent-1-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 vm1-eth0
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-1-vm2
Create VM agent-1-vm2
+ sudo ip netns add agent-1-vm2
+ sudo ip netns exec agent-1-vm2 ip l set lo up
+ sudo ip netns exec agent-1 ip link add vm2-eth0 type veth peer name eth0 netns agent-1-vm2
+ sudo ip netns exec agent-1 ip link set vm2-eth0 up
+ sudo ip netns exec agent-1-vm2 ip link set eth0 up
+ sudo ip netns exec agent-1-vm2 ip address add 10.0.1.2/16 dev eth0
+ sudo ip netns exec agent-1-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 vm2-eth0
+ touch /tmp/skydive-scale/agent-1.lock
+ '[' -z '' ']'
+ AGENTS=192.168.50.1
+ WINDOW=4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-2.lock ']'
+ create_host agent-2 192.168.50.2
+ NAME=agent-2
+ ADDR=192.168.50.2
+ echo Create host agent-2
Create host agent-2
+ sudo ip netns add agent-2
+ sudo ip netns exec agent-2 ip link set lo up
+ sudo ip link add agent-2-eth0 type veth peer name eth0 netns agent-2
+ sudo ip link set agent-2-eth0 up
+ sudo ip netns exec agent-2 ip link set eth0 up
+ sudo ip netns exec agent-2 ip address add 192.168.50.2/24 dev eth0
+ sudo ip netns exec agent-2 ip route add 0.0.0.0/0 via 192.168.50.254
+ sudo ip netns exec agent-2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-2-eth0
+ create_agent 2 192.168.50.2 2 2
+ IDX=2
+ ADDR=192.168.50.2
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-2
+ echo Create agent agent-2
Create agent agent-2
+ sudo ip netns exec agent-2 ovsdb-tool create /tmp/skydive-scale/agent-2.db
+ sudo ip netns exec agent-2 ovsdb-server /tmp/skydive-scale/agent-2.db --remote=punix:/tmp/skydive-scale/agent-2.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-2-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-2-dbserver.pid --detach
2019-05-15T16:08:46Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-2-vswitchd.log
+ sudo ip netns exec agent-2 ovs-vswitchd unix:/tmp/skydive-scale/agent-2.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-2-vswitchd.log --pidfile=/tmp/skydive-scale/agent-2-vswitchd.pid --detach --monitor
+ mkdir -p /tmp/skydive-scale/agent-2-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-2.cover
+ PIDFILE=/tmp/skydive-scale/agent-2.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 4 ip netns exec agent-2 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-2.cover ; export PIDFILE=/tmp/skydive-scale/agent-2.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-2.yml 2>&1 | tee /tmp/skydive-scale/agent-2.log'
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-br agent-2
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock set bridge agent-2 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-2-vm1
Create VM agent-2-vm1
+ sudo ip netns add agent-2-vm1
+ sudo ip netns exec agent-2-vm1 ip l set lo up
+ sudo ip netns exec agent-2 ip link add vm1-eth0 type veth peer name eth0 netns agent-2-vm1
+ sudo ip netns exec agent-2 ip link set vm1-eth0 up
+ sudo ip netns exec agent-2-vm1 ip link set eth0 up
+ sudo ip netns exec agent-2-vm1 ip address add 10.0.2.1/16 dev eth0
+ sudo ip netns exec agent-2-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 vm1-eth0
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-2-vm2
Create VM agent-2-vm2
+ sudo ip netns add agent-2-vm2
+ sudo ip netns exec agent-2-vm2 ip l set lo up
+ sudo ip netns exec agent-2 ip link add vm2-eth0 type veth peer name eth0 netns agent-2-vm2
+ sudo ip netns exec agent-2 ip link set vm2-eth0 up
+ sudo ip netns exec agent-2-vm2 ip link set eth0 up
+ sudo ip netns exec agent-2-vm2 ip address add 10.0.2.2/16 dev eth0
+ sudo ip netns exec agent-2-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 vm2-eth0
+ touch /tmp/skydive-scale/agent-2.lock
+ '[' -z 192.168.50.1 ']'
+ AGENTS=192.168.50.1,192.168.50.2
+ WINDOW=5
++ seq 2
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-1 192.168.50.1 192.168.50.1,192.168.50.2
+ NAME=agent-1
+ ADDR=192.168.50.1
+ AGENTS=192.168.50.1,192.168.50.2
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.1 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.2 ']'
+ echo '192.168.50.1 -> 192.168.50.2'
192.168.50.1 -> 192.168.50.2
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-2 192.168.50.2 192.168.50.1,192.168.50.2
+ NAME=agent-2
+ ADDR=192.168.50.2
+ AGENTS=192.168.50.1,192.168.50.2
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.1 ']'
+ echo '192.168.50.2 -> 192.168.50.1'
192.168.50.2 -> 192.168.50.1
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.2 ']'
+ echo Go to : http://192.168.50.254:8082
Go to : http://192.168.50.254:8082

2019-05-15T16:08:48.175Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [sleep 30]
2019-05-15T16:09:18.178Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned 
2019-05-15T16:09:18.499Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:18.641Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=2.06 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.055/2.055/2.055/0.000 ms

2019-05-15T16:09:19.641Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:19.804Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.176 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.176/0.176/0.176/0.000 ms

2019-05-15T16:09:20.804Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:20.928Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.159 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.159/0.159/0.159/0.000 ms

2019-05-15T16:09:21.928Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:22.043Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.109 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.109/0.109/0.109/0.000 ms

2019-05-15T16:09:23.044Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:23.183Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.122 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.122/0.122/0.122/0.000 ms

2019-05-15T16:09:24.183Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:24.316Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.190 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.190/0.190/0.190/0.000 ms

2019-05-15T16:09:25.317Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:25.470Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.191 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.191/0.191/0.191/0.000 ms

2019-05-15T16:09:26.470Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:26.596Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.147 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.147/0.147/0.147/0.000 ms

2019-05-15T16:09:27.597Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:27.709Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.119 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.119/0.119/0.119/0.000 ms

2019-05-15T16:09:28.709Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:28.837Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.159 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.159/0.159/0.159/0.000 ms

2019-05-15T16:09:29.838Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:29.957Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.125 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.125/0.125/0.125/0.000 ms

2019-05-15T16:09:30.958Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:31.115Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.149 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.149/0.149/0.149/0.000 ms

2019-05-15T16:09:32.115Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:32.234Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.146 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.146/0.146/0.146/0.000 ms

2019-05-15T16:09:33.235Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:33.361Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.392 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.392/0.392/0.392/0.000 ms

2019-05-15T16:09:34.362Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:34.494Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.130 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.130/0.130/0.130/0.000 ms

2019-05-15T16:09:35.495Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:35.628Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.121 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.121/0.121/0.121/0.000 ms

2019-05-15T16:09:36.628Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:36.744Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.142 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.142/0.142/0.142/0.000 ms

2019-05-15T16:09:37.744Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:37.889Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.199 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.199/0.199/0.199/0.000 ms

2019-05-15T16:09:38.891Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:39.015Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.132 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.132/0.132/0.132/0.000 ms

2019-05-15T16:09:40.016Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:40.140Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.228 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.228/0.228/0.228/0.000 ms

2019-05-15T16:09:41.140Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:41.265Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.138 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.138/0.138/0.138/0.000 ms

2019-05-15T16:09:42.266Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:42.391Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.137 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.137/0.137/0.137/0.000 ms

2019-05-15T16:09:43.392Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:43.527Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
++ sudo ip netns exec agent-2-vm1 ip a show eth0
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.126 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.126/0.126/0.126/0.000 ms

2019-05-15T16:09:44.528Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:44.647Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.135 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.135/0.135/0.135/0.000 ms

2019-05-15T16:09:45.648Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:45.789Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.152 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.152/0.152/0.152/0.000 ms

2019-05-15T16:09:46.790Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:46.951Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.145 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.145/0.145/0.145/0.000 ms

2019-05-15T16:09:47.951Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:48.098Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.135 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.135/0.135/0.135/0.000 ms

2019-05-15T16:09:49.098Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:49.227Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
++ sudo ip netns exec agent-2-vm1 ip a show eth0
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.114 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.114/0.114/0.114/0.000 ms

2019-05-15T16:09:50.228Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:50.376Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.128 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.128/0.128/0.128/0.000 ms

2019-05-15T16:09:51.377Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:09:51.506Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.116 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.116/0.116/0.116/0.000 ms

2019-05-15T16:10:03.319Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh start 2 4 2]
2019-05-15T16:10:09.778Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' start == start ']'
+ '[' -z 2 ']'
+ start 2 4 2
+ ANALYZER_NUM=2
+ AGENT_NUM=4
+ VM_NUM=2
+ '[' '!' -e /tmp/skydive-scale/screen.lock ']'
+ '[' true = true ']'
+ generate_ssl_conf 2 4
+ ANALYZER_NUM=2
+ AGENT_NUM=4
+ cat
++ seq 2
+ for I in $( seq $ANALYZER_NUM )
+ IDX=4
+ echo IP.4 = 192.168.50.1
+ for I in $( seq $ANALYZER_NUM )
+ IDX=5
+ echo IP.5 = 192.168.50.2
+ TOTAL_AGENT=9
++ seq 9
+ for I in $( seq $TOTAL_AGENT )
+ IP=3
+ IDX=5
+ echo IP.5 = 192.168.50.3
+ for I in $( seq $TOTAL_AGENT )
+ IP=4
+ IDX=6
+ echo IP.6 = 192.168.50.4
+ for I in $( seq $TOTAL_AGENT )
+ IP=5
+ IDX=7
+ echo IP.7 = 192.168.50.5
+ for I in $( seq $TOTAL_AGENT )
+ IP=6
+ IDX=8
+ echo IP.8 = 192.168.50.6
+ for I in $( seq $TOTAL_AGENT )
+ IP=7
+ IDX=9
+ echo IP.9 = 192.168.50.7
+ for I in $( seq $TOTAL_AGENT )
+ IP=8
+ IDX=10
+ echo IP.10 = 192.168.50.8
+ for I in $( seq $TOTAL_AGENT )
+ IP=9
+ IDX=11
+ echo IP.11 = 192.168.50.9
+ for I in $( seq $TOTAL_AGENT )
+ IP=10
+ IDX=12
+ echo IP.12 = 192.168.50.10
+ for I in $( seq $TOTAL_AGENT )
+ IP=11
+ IDX=13
+ echo IP.13 = 192.168.50.11
+ generate_tls_crt analyzer
+ NAME=analyzer
+ '[' -e /tmp/skydive-scale/analyzer.crt ']'
+ return
+ generate_tls_crt agent
+ NAME=agent
+ '[' -e /tmp/skydive-scale/agent.crt ']'
+ return
+ create_main_bridge
+ echo Create central bridge
Create central bridge
+ sudo brctl addbr br-central
device br-central already exists; can't create bridge with the same name
+ sudo brctl stp br-central off
+ sudo ip l set br-central up
+ sudo ip a add 192.168.50.254/24 dev br-central
RTNETLINK answers: File exists
+ start_inotify
+ '[' -f /tmp/skydive-scale/inotify.lock ']'
+ return
+ WINDOW=1
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 1 -gt 1 ']'
+ '[' '!' -f /tmp/skydive-scale/analyzer-1.lock ']'
+ CURR_ANALYZER_PORT=8084
+ WINDOW=2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 2 -gt 1 ']'
+ ETCD_EMBEDDED=false
+ '[' '!' -f /tmp/skydive-scale/analyzer-2.lock ']'
+ CURR_ANALYZER_PORT=8086
+ WINDOW=3
+ AGENTS=
++ seq 4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-1.lock ']'
+ '[' -z '' ']'
+ AGENTS=192.168.50.1
+ WINDOW=4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-2.lock ']'
+ '[' -z 192.168.50.1 ']'
+ AGENTS=192.168.50.1,192.168.50.2
+ WINDOW=5
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-3.lock ']'
+ create_host agent-3 192.168.50.3
+ NAME=agent-3
+ ADDR=192.168.50.3
+ echo Create host agent-3
Create host agent-3
+ sudo ip netns add agent-3
+ sudo ip netns exec agent-3 ip link set lo up
+ sudo ip link add agent-3-eth0 type veth peer name eth0 netns agent-3
+ sudo ip link set agent-3-eth0 up
+ sudo ip netns exec agent-3 ip link set eth0 up
+ sudo ip netns exec agent-3 ip address add 192.168.50.3/24 dev eth0
+ sudo ip netns exec agent-3 ip route add 0.0.0.0/0 via 192.168.50.254
+ sudo ip netns exec agent-3 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-3-eth0
+ create_agent 3 192.168.50.3 2 2
+ IDX=3
+ ADDR=192.168.50.3
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-3
+ echo Create agent agent-3
Create agent agent-3
+ sudo ip netns exec agent-3 ovsdb-tool create /tmp/skydive-scale/agent-3.db
+ sudo ip netns exec agent-3 ovsdb-server /tmp/skydive-scale/agent-3.db --remote=punix:/tmp/skydive-scale/agent-3.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-3-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-3-dbserver.pid --detach
2019-05-15T16:10:04Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-3-vswitchd.log
+ sudo ip netns exec agent-3 ovs-vswitchd unix:/tmp/skydive-scale/agent-3.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-3-vswitchd.log --pidfile=/tmp/skydive-scale/agent-3-vswitchd.pid --detach --monitor
+ mkdir -p /tmp/skydive-scale/agent-3-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-3.cover
+ PIDFILE=/tmp/skydive-scale/agent-3.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 5 ip netns exec agent-3 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-3.cover ; export PIDFILE=/tmp/skydive-scale/agent-3.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-3.yml 2>&1 | tee /tmp/skydive-scale/agent-3.log'
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-br agent-3
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock set bridge agent-3 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-3-vm1
Create VM agent-3-vm1
+ sudo ip netns add agent-3-vm1
+ sudo ip netns exec agent-3-vm1 ip l set lo up
+ sudo ip netns exec agent-3 ip link add vm1-eth0 type veth peer name eth0 netns agent-3-vm1
+ sudo ip netns exec agent-3 ip link set vm1-eth0 up
+ sudo ip netns exec agent-3-vm1 ip link set eth0 up
+ sudo ip netns exec agent-3-vm1 ip address add 10.0.3.1/16 dev eth0
+ sudo ip netns exec agent-3-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 vm1-eth0
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-3-vm2
Create VM agent-3-vm2
+ sudo ip netns add agent-3-vm2
+ sudo ip netns exec agent-3-vm2 ip l set lo up
+ sudo ip netns exec agent-3 ip link add vm2-eth0 type veth peer name eth0 netns agent-3-vm2
+ sudo ip netns exec agent-3 ip link set vm2-eth0 up
+ sudo ip netns exec agent-3-vm2 ip link set eth0 up
+ sudo ip netns exec agent-3-vm2 ip address add 10.0.3.2/16 dev eth0
+ sudo ip netns exec agent-3-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 vm2-eth0
+ touch /tmp/skydive-scale/agent-3.lock
+ '[' -z 192.168.50.1,192.168.50.2 ']'
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ WINDOW=6
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-4.lock ']'
+ create_host agent-4 192.168.50.4
+ NAME=agent-4
+ ADDR=192.168.50.4
+ echo Create host agent-4
Create host agent-4
+ sudo ip netns add agent-4
+ sudo ip netns exec agent-4 ip link set lo up
+ sudo ip link add agent-4-eth0 type veth peer name eth0 netns agent-4
+ sudo ip link set agent-4-eth0 up
+ sudo ip netns exec agent-4 ip link set eth0 up
+ sudo ip netns exec agent-4 ip address add 192.168.50.4/24 dev eth0
+ sudo ip netns exec agent-4 ip route add 0.0.0.0/0 via 192.168.50.254
+ sudo ip netns exec agent-4 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-4-eth0
+ create_agent 4 192.168.50.4 2 2
+ IDX=4
+ ADDR=192.168.50.4
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-4
+ echo Create agent agent-4
Create agent agent-4
+ sudo ip netns exec agent-4 ovsdb-tool create /tmp/skydive-scale/agent-4.db
+ sudo ip netns exec agent-4 ovsdb-server /tmp/skydive-scale/agent-4.db --remote=punix:/tmp/skydive-scale/agent-4.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-4-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-4-dbserver.pid --detach
2019-05-15T16:10:06Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-4-vswitchd.log
+ sudo ip netns exec agent-4 ovs-vswitchd unix:/tmp/skydive-scale/agent-4.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-4-vswitchd.log --pidfile=/tmp/skydive-scale/agent-4-vswitchd.pid --detach --monitor
+ mkdir -p /tmp/skydive-scale/agent-4-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-4.cover
+ PIDFILE=/tmp/skydive-scale/agent-4.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 6 ip netns exec agent-4 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-4.cover ; export PIDFILE=/tmp/skydive-scale/agent-4.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-4.yml 2>&1 | tee /tmp/skydive-scale/agent-4.log'
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-br agent-4
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock set bridge agent-4 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-4-vm1
Create VM agent-4-vm1
+ sudo ip netns add agent-4-vm1
+ sudo ip netns exec agent-4-vm1 ip l set lo up
+ sudo ip netns exec agent-4 ip link add vm1-eth0 type veth peer name eth0 netns agent-4-vm1
+ sudo ip netns exec agent-4 ip link set vm1-eth0 up
+ sudo ip netns exec agent-4-vm1 ip link set eth0 up
+ sudo ip netns exec agent-4-vm1 ip address add 10.0.4.1/16 dev eth0
+ sudo ip netns exec agent-4-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 vm1-eth0
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-4-vm2
Create VM agent-4-vm2
+ sudo ip netns add agent-4-vm2
+ sudo ip netns exec agent-4-vm2 ip l set lo up
+ sudo ip netns exec agent-4 ip link add vm2-eth0 type veth peer name eth0 netns agent-4-vm2
+ sudo ip netns exec agent-4 ip link set vm2-eth0 up
+ sudo ip netns exec agent-4-vm2 ip link set eth0 up
+ sudo ip netns exec agent-4-vm2 ip address add 10.0.4.2/16 dev eth0
+ sudo ip netns exec agent-4-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 vm2-eth0
+ touch /tmp/skydive-scale/agent-4.lock
+ '[' -z 192.168.50.1,192.168.50.2,192.168.50.3 ']'
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ WINDOW=7
++ seq 4
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-1 192.168.50.1 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ NAME=agent-1
+ ADDR=192.168.50.1
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.1 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.2 ']'
+ echo '192.168.50.1 -> 192.168.50.2'
192.168.50.1 -> 192.168.50.2
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.3 ']'
+ echo '192.168.50.1 -> 192.168.50.3'
192.168.50.1 -> 192.168.50.3
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.4 ']'
+ echo '192.168.50.1 -> 192.168.50.4'
192.168.50.1 -> 192.168.50.4
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.4 -- set interface gre-192.168.50.4 type=gre options:remote_ip=192.168.50.4
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-2 192.168.50.2 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ NAME=agent-2
+ ADDR=192.168.50.2
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.1 ']'
+ echo '192.168.50.2 -> 192.168.50.1'
192.168.50.2 -> 192.168.50.1
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.2 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.3 ']'
+ echo '192.168.50.2 -> 192.168.50.3'
192.168.50.2 -> 192.168.50.3
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.4 ']'
+ echo '192.168.50.2 -> 192.168.50.4'
192.168.50.2 -> 192.168.50.4
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.4 -- set interface gre-192.168.50.4 type=gre options:remote_ip=192.168.50.4
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-3 192.168.50.3 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ NAME=agent-3
+ ADDR=192.168.50.3
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.1 ']'
+ echo '192.168.50.3 -> 192.168.50.1'
192.168.50.3 -> 192.168.50.1
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.2 ']'
+ echo '192.168.50.3 -> 192.168.50.2'
192.168.50.3 -> 192.168.50.2
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.3 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.4 ']'
+ echo '192.168.50.3 -> 192.168.50.4'
192.168.50.3 -> 192.168.50.4
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.4 -- set interface gre-192.168.50.4 type=gre options:remote_ip=192.168.50.4
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-4 192.168.50.4 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ NAME=agent-4
+ ADDR=192.168.50.4
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3,192.168.50.4
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.4 '!=' 192.168.50.1 ']'
+ echo '192.168.50.4 -> 192.168.50.1'
192.168.50.4 -> 192.168.50.1
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.4 '!=' 192.168.50.2 ']'
+ echo '192.168.50.4 -> 192.168.50.2'
192.168.50.4 -> 192.168.50.2
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.4 '!=' 192.168.50.3 ']'
+ echo '192.168.50.4 -> 192.168.50.3'
192.168.50.4 -> 192.168.50.3
+ sudo ip netns exec agent-4 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-4.sock add-port agent-4 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.4 '!=' 192.168.50.4 ']'
+ echo Go to : http://192.168.50.254:8082
Go to : http://192.168.50.254:8082

2019-05-15T16:10:10.019Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh stop-agent 4]
2019-05-15T16:10:10.363Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop-agent == start ']'
+ '[' stop-agent == stop ']'
+ '[' stop-agent == stop-analyzer ']'
+ '[' stop-agent == stop-agent ']'
+ '[' -z 4 ']'
+ stop_agent 4
+ AGENT=4
+ NAME=agent-4
+ '[' -e /tmp/skydive-scale/agent-4.pid ']'
+ sudo pkill -f agent-4.yml
+ timeout=30
+ pkill -0 -f agent-4.yml
+ sudo rm /tmp/skydive-scale/agent-4.lock /tmp/skydive-scale/agent-4.pid
rm: cannot remove '/tmp/skydive-scale/agent-4.pid': No such file or directory

2019-05-15T16:10:10.754Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh stop-analyzer 2]
2019-05-15T16:10:11.085Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop-analyzer == start ']'
+ '[' stop-analyzer == stop ']'
+ '[' stop-analyzer == stop-analyzer ']'
+ '[' -z 2 ']'
+ stop_analyzer 2
+ ANALYZER=2
+ NAME=analyzer-2
+ '[' -e /tmp/skydive-scale/analyzer-2.pid ']'
+ sudo pkill -f analyzer-2.yml
+ timeout=30
+ pkill -0 -f analyzer-2.yml
+ sudo rm /tmp/skydive-scale/analyzer-2.lock /tmp/skydive-scale/analyzer-2.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-2.pid': No such file or directory

2019-05-15T16:10:11.085Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [sleep 5]
2019-05-15T16:10:16.087Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned 
2019-05-15T16:10:16.196Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:16.325Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=2.53 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.529/2.529/2.529/0.000 ms

2019-05-15T16:10:17.326Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:17.450Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.142 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.142/0.142/0.142/0.000 ms

2019-05-15T16:10:18.451Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:18.569Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.208 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.208/0.208/0.208/0.000 ms

2019-05-15T16:10:19.569Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:19.694Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.152 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.152/0.152/0.152/0.000 ms

2019-05-15T16:10:20.694Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:20.836Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.209 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.209/0.209/0.209/0.000 ms

2019-05-15T16:10:21.836Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:21.978Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.142 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.142/0.142/0.142/0.000 ms

2019-05-15T16:10:22.979Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:23.114Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.392 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.392/0.392/0.392/0.000 ms

2019-05-15T16:10:24.115Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:24.237Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.454 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.454/0.454/0.454/0.000 ms

2019-05-15T16:10:25.238Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:25.373Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.168 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.168/0.168/0.168/0.000 ms

2019-05-15T16:10:26.374Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:26.512Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.149 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.149/0.149/0.149/0.000 ms

2019-05-15T16:10:27.513Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:27.637Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
++ sudo ip netns exec agent-1-vm1 ip a show eth0
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.153 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.153/0.153/0.153/0.000 ms

2019-05-15T16:10:28.638Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:28.762Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.178 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.178/0.178/0.178/0.000 ms

2019-05-15T16:10:29.763Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:29.878Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.164 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.164/0.164/0.164/0.000 ms

2019-05-15T16:10:30.879Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:31.035Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ cut -d / -f 1
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.195 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.195/0.195/0.195/0.000 ms

2019-05-15T16:10:32.035Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:32.152Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.159 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.159/0.159/0.159/0.000 ms

2019-05-15T16:10:33.152Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:33.266Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.181 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.181/0.181/0.181/0.000 ms

2019-05-15T16:10:34.266Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:34.407Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ awk '{print $2}'
++ cut -d / -f 1
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.223 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.223/0.223/0.223/0.000 ms

2019-05-15T16:10:35.408Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:35.608Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ cut -d / -f 1
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.260 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.260/0.260/0.260/0.000 ms

2019-05-15T16:10:36.608Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:36.731Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.141 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.141/0.141/0.141/0.000 ms

2019-05-15T16:10:37.732Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:37.873Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.160 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.160/0.160/0.160/0.000 ms

2019-05-15T16:10:38.873Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:38.994Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.151 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.151/0.151/0.151/0.000 ms

2019-05-15T16:10:39.994Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:40.123Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.142 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.142/0.142/0.142/0.000 ms

2019-05-15T16:10:41.123Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:41.252Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.168 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.168/0.168/0.168/0.000 ms

2019-05-15T16:10:42.252Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:42.374Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.142 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.142/0.142/0.142/0.000 ms

2019-05-15T16:10:43.374Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:43.507Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ awk '{print $2}'
++ cut -d / -f 1
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.186 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.186/0.186/0.186/0.000 ms

2019-05-15T16:10:44.508Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:44.626Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.179 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.179/0.179/0.179/0.000 ms

2019-05-15T16:10:45.627Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:45.801Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.345 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.345/0.345/0.345/0.000 ms

2019-05-15T16:10:46.801Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:46.949Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.151 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.151/0.151/0.151/0.000 ms

2019-05-15T16:10:47.949Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:48.078Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.230 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.230/0.230/0.230/0.000 ms

2019-05-15T16:10:49.079Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-3-vm1 agent-1-vm1 -c 1]
2019-05-15T16:10:49.203Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-1-vm1 ']'
+ ping agent-3-vm1 agent-1-vm1 -c 1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sudo ip netns exec agent-3-vm1 ping 10.0.1.1 -c 1
PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data.
64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.171 ms

--- 10.0.1.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.171/0.171/0.171/0.000 ms

2019-05-15T16:11:01.798Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh iperf agent-3-vm1 agent-1-vm1]
2019-05-15T16:11:14.743Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' iperf == start ']'
+ '[' iperf == stop ']'
+ '[' iperf == stop-analyzer ']'
+ '[' iperf == stop-agent ']'
+ '[' iperf == ping ']'
+ '[' iperf == iperf ']'
+ '[' -z agent-1-vm1 ']'
+ iperf agent-3-vm1 agent-1-vm1
+ SRC=agent-3-vm1
+ DST=agent-1-vm1
+ ARGS=
++ sudo ip netns exec agent-1-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.1.1
+ sleep 1
+ sudo ip netns exec agent-1-vm1 iperf -s -t 11
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------
+ sudo ip netns exec agent-3-vm1 iperf -c 10.0.1.1 -t 10 -b 1M
------------------------------------------------------------
Client connecting to 10.0.1.1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.3.1 port 35564 connected with 10.0.1.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.4 sec   107 KBytes  84.6 Kbits/sec
+ sudo pkill -f 'iperf -s'
+ sleep 1
+ sudo pkill -f 'iperf -s'

2019-05-15T16:11:17.713Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh start 2 3 2]
2019-05-15T16:11:18.646Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' start == start ']'
+ '[' -z 2 ']'
+ start 2 3 2
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ VM_NUM=2
+ '[' '!' -e /tmp/skydive-scale/screen.lock ']'
+ '[' true = true ']'
+ generate_ssl_conf 2 3
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ cat
++ seq 2
+ for I in $( seq $ANALYZER_NUM )
+ IDX=4
+ echo IP.4 = 192.168.50.1
+ for I in $( seq $ANALYZER_NUM )
+ IDX=5
+ echo IP.5 = 192.168.50.2
+ TOTAL_AGENT=8
++ seq 8
+ for I in $( seq $TOTAL_AGENT )
+ IP=3
+ IDX=5
+ echo IP.5 = 192.168.50.3
+ for I in $( seq $TOTAL_AGENT )
+ IP=4
+ IDX=6
+ echo IP.6 = 192.168.50.4
+ for I in $( seq $TOTAL_AGENT )
+ IP=5
+ IDX=7
+ echo IP.7 = 192.168.50.5
+ for I in $( seq $TOTAL_AGENT )
+ IP=6
+ IDX=8
+ echo IP.8 = 192.168.50.6
+ for I in $( seq $TOTAL_AGENT )
+ IP=7
+ IDX=9
+ echo IP.9 = 192.168.50.7
+ for I in $( seq $TOTAL_AGENT )
+ IP=8
+ IDX=10
+ echo IP.10 = 192.168.50.8
+ for I in $( seq $TOTAL_AGENT )
+ IP=9
+ IDX=11
+ echo IP.11 = 192.168.50.9
+ for I in $( seq $TOTAL_AGENT )
+ IP=10
+ IDX=12
+ echo IP.12 = 192.168.50.10
+ generate_tls_crt analyzer
+ NAME=analyzer
+ '[' -e /tmp/skydive-scale/analyzer.crt ']'
+ return
+ generate_tls_crt agent
+ NAME=agent
+ '[' -e /tmp/skydive-scale/agent.crt ']'
+ return
+ create_main_bridge
+ echo Create central bridge
Create central bridge
+ sudo brctl addbr br-central
device br-central already exists; can't create bridge with the same name
+ sudo brctl stp br-central off
+ sudo ip l set br-central up
+ sudo ip a add 192.168.50.254/24 dev br-central
RTNETLINK answers: File exists
+ start_inotify
+ '[' -f /tmp/skydive-scale/inotify.lock ']'
+ return
+ WINDOW=1
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 1 -gt 1 ']'
+ '[' '!' -f /tmp/skydive-scale/analyzer-1.lock ']'
+ CURR_ANALYZER_PORT=8084
+ WINDOW=2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 2 -gt 1 ']'
+ ETCD_EMBEDDED=false
+ '[' '!' -f /tmp/skydive-scale/analyzer-2.lock ']'
+ create_analyzer 2 2 3
+ IDX=2
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ NAME=analyzer-2
+ echo Create analyzer analyzer-2
Create analyzer analyzer-2
+ mkdir -p /tmp/skydive-scale/analyzer-2-etcd
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - localhost:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - localhost:8084'
+ STORAGE=
+ GRAPH=memory
+ '[' xlocalhost:9200 '!=' x ']'
+ STORAGE=elasticsearch
+ GRAPH=elasticsearch
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ cat
+ TOTAL_AGENT=8
++ seq 8
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_1'
+ echo '      - TOR_PORT_1 --> *[Name=agent-1]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_2'
+ echo '      - TOR_PORT_2 --> *[Name=agent-2]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_3'
+ echo '      - TOR_PORT_3 --> *[Name=agent-3]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_4'
+ echo '      - TOR_PORT_4 --> *[Name=agent-4]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_5'
+ echo '      - TOR_PORT_5 --> *[Name=agent-5]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_6'
+ echo '      - TOR_PORT_6 --> *[Name=agent-6]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_7'
+ echo '      - TOR_PORT_7 --> *[Name=agent-7]/eth0'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      - TOR -> TOR_PORT_8'
+ echo '      - TOR_PORT_8 --> *[Name=agent-8]/eth0'
+ cat
++ seq 8
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-1: agent-password-1'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-2: agent-password-2'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-3: agent-password-3'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-4: agent-password-4'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-5: agent-password-5'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-6: agent-password-6'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-7: agent-password-7'
+ for AGENT_I in $( seq $TOTAL_AGENT )
+ echo '      agent-8: agent-password-8'
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-1: analyzer-password-1'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ echo '      analyzer-2: analyzer-password-2'
+ COVERFILE=/tmp/skydive-scale/analyzer-2.cover
+ PIDFILE=/tmp/skydive-scale/analyzer-2.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-analyzer 2 sh -c 'export COVERFILE=/tmp/skydive-scale/analyzer-2.cover ; export PIDFILE=/tmp/skydive-scale/analyzer-2.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive analyzer -c /tmp/skydive-scale/analyzer-2.yml 2>&1 | tee /tmp/skydive-scale/analyzer-2.log'
+ touch /tmp/skydive-scale/analyzer-2.lock
+ CURR_ANALYZER_PORT=8086
+ WINDOW=3
+ AGENTS=
++ seq 3
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-1.lock ']'
+ '[' -z '' ']'
+ AGENTS=192.168.50.1
+ WINDOW=4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-2.lock ']'
+ '[' -z 192.168.50.1 ']'
+ AGENTS=192.168.50.1,192.168.50.2
+ WINDOW=5
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-3.lock ']'
+ '[' -z 192.168.50.1,192.168.50.2 ']'
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ WINDOW=6
++ seq 3
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-1 192.168.50.1 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-1
+ ADDR=192.168.50.1
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ sed 's/,/ /g'
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.1 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.2 ']'
+ echo '192.168.50.1 -> 192.168.50.2'
192.168.50.1 -> 192.168.50.2
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.3 ']'
+ echo '192.168.50.1 -> 192.168.50.3'
192.168.50.1 -> 192.168.50.3
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
ovs-vsctl: cannot create a port named gre-192.168.50.3 because a port named gre-192.168.50.3 already exists on bridge agent-1
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-2 192.168.50.2 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-2
+ ADDR=192.168.50.2
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ sed 's/,/ /g'
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.1 ']'
+ echo '192.168.50.2 -> 192.168.50.1'
192.168.50.2 -> 192.168.50.1
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.2 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.3 ']'
+ echo '192.168.50.2 -> 192.168.50.3'
192.168.50.2 -> 192.168.50.3
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
ovs-vsctl: cannot create a port named gre-192.168.50.3 because a port named gre-192.168.50.3 already exists on bridge agent-2
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-3 192.168.50.3 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-3
+ ADDR=192.168.50.3
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ sed 's/,/ /g'
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.1 ']'
+ echo '192.168.50.3 -> 192.168.50.1'
192.168.50.3 -> 192.168.50.1
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.2 ']'
+ echo '192.168.50.3 -> 192.168.50.2'
192.168.50.3 -> 192.168.50.2
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.3 ']'
+ echo Go to : http://192.168.50.254:8082
Go to : http://192.168.50.254:8082

2019-05-15T16:11:18.646Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [sleep 5]
2019-05-15T16:11:23.648Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned 
2019-05-15T16:11:23.652Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh stop-agent 1]
2019-05-15T16:11:23.908Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop-agent == start ']'
+ '[' stop-agent == stop ']'
+ '[' stop-agent == stop-analyzer ']'
+ '[' stop-agent == stop-agent ']'
+ '[' -z 1 ']'
+ stop_agent 1
+ AGENT=1
+ NAME=agent-1
+ '[' -e /tmp/skydive-scale/agent-1.pid ']'
+ sudo pkill -f agent-1.yml
+ timeout=30
+ pkill -0 -f agent-1.yml
+ sudo rm /tmp/skydive-scale/agent-1.lock /tmp/skydive-scale/agent-1.pid
rm: cannot remove '/tmp/skydive-scale/agent-1.pid': No such file or directory

2019-05-15T16:11:23.991Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh start 2 3 2]
2019-05-15T16:11:26.797Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' start == start ']'
+ '[' -z 2 ']'
+ start 2 3 2
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ VM_NUM=2
+ '[' '!' -e /tmp/skydive-scale/screen.lock ']'
+ '[' true = true ']'
+ generate_ssl_conf 2 3
+ ANALYZER_NUM=2
+ AGENT_NUM=3
+ cat
++ seq 2
+ for I in $( seq $ANALYZER_NUM )
+ IDX=4
+ echo IP.4 = 192.168.50.1
+ for I in $( seq $ANALYZER_NUM )
+ IDX=5
+ echo IP.5 = 192.168.50.2
+ TOTAL_AGENT=8
++ seq 8
+ for I in $( seq $TOTAL_AGENT )
+ IP=3
+ IDX=5
+ echo IP.5 = 192.168.50.3
+ for I in $( seq $TOTAL_AGENT )
+ IP=4
+ IDX=6
+ echo IP.6 = 192.168.50.4
+ for I in $( seq $TOTAL_AGENT )
+ IP=5
+ IDX=7
+ echo IP.7 = 192.168.50.5
+ for I in $( seq $TOTAL_AGENT )
+ IP=6
+ IDX=8
+ echo IP.8 = 192.168.50.6
+ for I in $( seq $TOTAL_AGENT )
+ IP=7
+ IDX=9
+ echo IP.9 = 192.168.50.7
+ for I in $( seq $TOTAL_AGENT )
+ IP=8
+ IDX=10
+ echo IP.10 = 192.168.50.8
+ for I in $( seq $TOTAL_AGENT )
+ IP=9
+ IDX=11
+ echo IP.11 = 192.168.50.9
+ for I in $( seq $TOTAL_AGENT )
+ IP=10
+ IDX=12
+ echo IP.12 = 192.168.50.10
+ generate_tls_crt analyzer
+ NAME=analyzer
+ '[' -e /tmp/skydive-scale/analyzer.crt ']'
+ return
+ generate_tls_crt agent
+ NAME=agent
+ '[' -e /tmp/skydive-scale/agent.crt ']'
+ return
+ create_main_bridge
+ echo Create central bridge
Create central bridge
+ sudo brctl addbr br-central
device br-central already exists; can't create bridge with the same name
+ sudo brctl stp br-central off
+ sudo ip l set br-central up
+ sudo ip a add 192.168.50.254/24 dev br-central
RTNETLINK answers: File exists
+ start_inotify
+ '[' -f /tmp/skydive-scale/inotify.lock ']'
+ return
+ WINDOW=1
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 1 -gt 1 ']'
+ '[' '!' -f /tmp/skydive-scale/analyzer-1.lock ']'
+ CURR_ANALYZER_PORT=8084
+ WINDOW=2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ '[' 2 -gt 1 ']'
+ ETCD_EMBEDDED=false
+ '[' '!' -f /tmp/skydive-scale/analyzer-2.lock ']'
+ CURR_ANALYZER_PORT=8086
+ WINDOW=3
+ AGENTS=
++ seq 3
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-1.lock ']'
+ create_host agent-1 192.168.50.1
+ NAME=agent-1
+ ADDR=192.168.50.1
+ echo Create host agent-1
Create host agent-1
+ sudo ip netns add agent-1
Cannot create namespace file "/var/run/netns/agent-1": File exists
+ sudo ip netns exec agent-1 ip link set lo up
+ sudo ip link add agent-1-eth0 type veth peer name eth0 netns agent-1
RTNETLINK answers: File exists
+ sudo ip link set agent-1-eth0 up
+ sudo ip netns exec agent-1 ip link set eth0 up
+ sudo ip netns exec agent-1 ip address add 192.168.50.1/24 dev eth0
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1 ip route add 0.0.0.0/0 via 192.168.50.254
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo brctl addif br-central agent-1-eth0
device agent-1-eth0 is already a member of a bridge; can't enslave it to bridge br-central.
+ create_agent 1 192.168.50.1 2 2
+ IDX=1
+ ADDR=192.168.50.1
+ ANALYZER_NUM=2
+ VM_NUM=2
+ NAME=agent-1
+ echo Create agent agent-1
Create agent agent-1
+ sudo ip netns exec agent-1 ovsdb-tool create /tmp/skydive-scale/agent-1.db
2019-05-15T16:11:24Z|00001|lockfile|WARN|/tmp/skydive-scale/.agent-1.db.~lock~: cannot lock file because it is already locked by pid 19730
ovsdb-tool: I/O error: /tmp/skydive-scale/agent-1.db: failed to lock lockfile (Resource temporarily unavailable)
+ sudo ip netns exec agent-1 ovsdb-server /tmp/skydive-scale/agent-1.db --remote=punix:/tmp/skydive-scale/agent-1.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --log-file=/tmp/skydive-scale/agent-1-vswitchd.log -vsyslog:dbg -vfile:dbg --pidfile=/tmp/skydive-scale/agent-1-dbserver.pid --detach
2019-05-15T16:11:24Z|00001|vlog|INFO|opened log file /tmp/skydive-scale/agent-1-vswitchd.log
ovsdb-server: /tmp/skydive-scale/agent-1-dbserver.pid: already running as pid 19730, aborting
+ sudo ip netns exec agent-1 ovs-vswitchd unix:/tmp/skydive-scale/agent-1.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/tmp/skydive-scale/agent-1-vswitchd.log --pidfile=/tmp/skydive-scale/agent-1-vswitchd.pid --detach --monitor
ovs-vswitchd: /tmp/skydive-scale/agent-1-vswitchd.pid: already running as pid 19737, aborting
+ mkdir -p /tmp/skydive-scale/agent-1-netns
+ '[' true = true ']'
+ CA_CRT=/tmp/skydive-scale/rootCA.crt
+ AGENT_CRT=/tmp/skydive-scale/agent.crt
+ AGENT_KEY=/tmp/skydive-scale/agent.key
+ ANALYZER_CRT=/tmp/skydive-scale/analyzer.crt
+ ANALYZER_KEY=/tmp/skydive-scale/analyzer.key
+ echo analyzers:
++ seq 2
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8082
+ echo '  - 192.168.50.254:8082'
+ for ANALYZER_I in $( seq $ANALYZER_NUM )
+ PORT=8084
+ echo '  - 192.168.50.254:8084'
+ cat
+ COVERFILE=/tmp/skydive-scale/agent-1.cover
+ PIDFILE=/tmp/skydive-scale/agent-1.pid
+ sudo -E screen -S skydive-stress -X screen -t skydive-agent 3 ip netns exec agent-1 sh -c 'export COVERFILE=/tmp/skydive-scale/agent-1.cover ; export PIDFILE=/tmp/skydive-scale/agent-1.pid ; /var/lib/jenkins/workspace/skydive-scale-tests/bin/skydive agent -c /tmp/skydive-scale/agent-1.yml 2>&1 | tee /tmp/skydive-scale/agent-1.log'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-br agent-1
ovs-vsctl: cannot create a bridge named agent-1 because a bridge named agent-1 already exists
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock set bridge agent-1 rstp_enable=true
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-1-vm1
Create VM agent-1-vm1
+ sudo ip netns add agent-1-vm1
Cannot create namespace file "/var/run/netns/agent-1-vm1": File exists
+ sudo ip netns exec agent-1-vm1 ip l set lo up
+ sudo ip netns exec agent-1 ip link add vm1-eth0 type veth peer name eth0 netns agent-1-vm1
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1 ip link set vm1-eth0 up
+ sudo ip netns exec agent-1-vm1 ip link set eth0 up
+ sudo ip netns exec agent-1-vm1 ip address add 10.0.1.1/16 dev eth0
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1-vm1 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 vm1-eth0
ovs-vsctl: cannot create a port named vm1-eth0 because a port named vm1-eth0 already exists on bridge agent-1
+ for VM_I in $( seq $VM_NUM )
+ echo Create VM agent-1-vm2
Create VM agent-1-vm2
+ sudo ip netns add agent-1-vm2
Cannot create namespace file "/var/run/netns/agent-1-vm2": File exists
+ sudo ip netns exec agent-1-vm2 ip l set lo up
+ sudo ip netns exec agent-1 ip link add vm2-eth0 type veth peer name eth0 netns agent-1-vm2
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1 ip link set vm2-eth0 up
+ sudo ip netns exec agent-1-vm2 ip link set eth0 up
+ sudo ip netns exec agent-1-vm2 ip address add 10.0.1.2/16 dev eth0
RTNETLINK answers: File exists
+ sudo ip netns exec agent-1-vm2 bash -c 'echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts'
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 vm2-eth0
ovs-vsctl: cannot create a port named vm2-eth0 because a port named vm2-eth0 already exists on bridge agent-1
+ touch /tmp/skydive-scale/agent-1.lock
+ '[' -z '' ']'
+ AGENTS=192.168.50.1
+ WINDOW=4
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-2.lock ']'
+ '[' -z 192.168.50.1 ']'
+ AGENTS=192.168.50.1,192.168.50.2
+ WINDOW=5
+ for AGENT_I in $( seq $AGENT_NUM )
+ '[' '!' -f /tmp/skydive-scale/agent-3.lock ']'
+ '[' -z 192.168.50.1,192.168.50.2 ']'
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ WINDOW=6
++ seq 3
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-1 192.168.50.1 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-1
+ ADDR=192.168.50.1
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.1 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.2 ']'
+ echo '192.168.50.1 -> 192.168.50.2'
192.168.50.1 -> 192.168.50.2
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-1
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.1 '!=' 192.168.50.3 ']'
+ echo '192.168.50.1 -> 192.168.50.3'
192.168.50.1 -> 192.168.50.3
+ sudo ip netns exec agent-1 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-1.sock add-port agent-1 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
ovs-vsctl: cannot create a port named gre-192.168.50.3 because a port named gre-192.168.50.3 already exists on bridge agent-1
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-2 192.168.50.2 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-2
+ ADDR=192.168.50.2
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.1 ']'
+ echo '192.168.50.2 -> 192.168.50.1'
192.168.50.2 -> 192.168.50.1
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-2
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.2 ']'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.2 '!=' 192.168.50.3 ']'
+ echo '192.168.50.2 -> 192.168.50.3'
192.168.50.2 -> 192.168.50.3
+ sudo ip netns exec agent-2 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-2.sock add-port agent-2 gre-192.168.50.3 -- set interface gre-192.168.50.3 type=gre options:remote_ip=192.168.50.3
ovs-vsctl: cannot create a port named gre-192.168.50.3 because a port named gre-192.168.50.3 already exists on bridge agent-2
+ for AGENT_I in $( seq $AGENT_NUM )
+ create_tunnel agent-3 192.168.50.3 192.168.50.1,192.168.50.2,192.168.50.3
+ NAME=agent-3
+ ADDR=192.168.50.3
+ AGENTS=192.168.50.1,192.168.50.2,192.168.50.3
+ echo Create tunnels...
Create tunnels...
++ echo 192.168.50.1,192.168.50.2,192.168.50.3
++ sed 's/,/ /g'
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.1 ']'
+ echo '192.168.50.3 -> 192.168.50.1'
192.168.50.3 -> 192.168.50.1
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.1 -- set interface gre-192.168.50.1 type=gre options:remote_ip=192.168.50.1
ovs-vsctl: cannot create a port named gre-192.168.50.1 because a port named gre-192.168.50.1 already exists on bridge agent-3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.2 ']'
+ echo '192.168.50.3 -> 192.168.50.2'
192.168.50.3 -> 192.168.50.2
+ sudo ip netns exec agent-3 ovs-vsctl --db=unix:/tmp/skydive-scale/agent-3.sock add-port agent-3 gre-192.168.50.2 -- set interface gre-192.168.50.2 type=gre options:remote_ip=192.168.50.2
ovs-vsctl: cannot create a port named gre-192.168.50.2 because a port named gre-192.168.50.2 already exists on bridge agent-3
+ for AGENT_I in $( echo $AGENTS | sed "s/,/ /g" )
+ '[' 192.168.50.3 '!=' 192.168.50.3 ']'
+ echo Go to : http://192.168.50.254:8082
Go to : http://192.168.50.254:8082

2019-05-15T16:11:26.797Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [sleep 5]
2019-05-15T16:11:31.799Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned 
2019-05-15T16:11:32.035Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:32.162Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=1.79 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.794/1.794/1.794/0.000 ms

2019-05-15T16:11:33.163Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:33.284Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.172 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.172/0.172/0.172/0.000 ms

2019-05-15T16:11:34.284Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:34.434Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.235 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.235/0.235/0.235/0.000 ms

2019-05-15T16:11:35.435Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:35.576Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.156 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.156/0.156/0.156/0.000 ms

2019-05-15T16:11:36.577Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:36.699Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.259 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.259/0.259/0.259/0.000 ms

2019-05-15T16:11:37.699Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:37.819Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.150 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.150/0.150/0.150/0.000 ms

2019-05-15T16:11:38.820Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:38.948Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ grep 'inet '
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.211 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.211/0.211/0.211/0.000 ms

2019-05-15T16:11:39.949Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:40.099Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.215 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.215/0.215/0.215/0.000 ms

2019-05-15T16:11:41.100Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:41.240Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.247 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.247/0.247/0.247/0.000 ms

2019-05-15T16:11:42.240Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:42.358Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.149 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.149/0.149/0.149/0.000 ms

2019-05-15T16:11:43.358Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:43.488Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.161 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.161/0.161/0.161/0.000 ms

2019-05-15T16:11:44.489Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:44.629Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.151 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.151/0.151/0.151/0.000 ms

2019-05-15T16:11:45.630Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:45.773Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.184 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.184/0.184/0.184/0.000 ms

2019-05-15T16:11:46.773Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:46.906Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.253 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.253/0.253/0.253/0.000 ms

2019-05-15T16:11:47.907Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:48.050Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ grep 'inet '
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.180 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.180/0.180/0.180/0.000 ms

2019-05-15T16:11:49.051Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:49.183Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.154 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.154/0.154/0.154/0.000 ms

2019-05-15T16:11:50.183Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:50.308Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.217 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.217/0.217/0.217/0.000 ms

2019-05-15T16:11:51.308Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:51.441Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.142 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.142/0.142/0.142/0.000 ms

2019-05-15T16:11:52.442Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:52.577Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.146 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.146/0.146/0.146/0.000 ms

2019-05-15T16:11:53.578Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:53.707Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.154 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.154/0.154/0.154/0.000 ms

2019-05-15T16:11:54.707Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:54.834Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.235 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.235/0.235/0.235/0.000 ms

2019-05-15T16:11:55.834Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:55.999Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.228 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.228/0.228/0.228/0.000 ms

2019-05-15T16:11:57.000Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:57.118Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.134 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.134/0.134/0.134/0.000 ms

2019-05-15T16:11:58.119Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:58.241Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.182 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.182/0.182/0.182/0.000 ms

2019-05-15T16:11:59.242Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:11:59.391Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
++ sudo ip netns exec agent-2-vm1 ip a show eth0
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.201 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.201/0.201/0.201/0.000 ms

2019-05-15T16:12:00.391Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:12:00.542Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.170 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.170/0.170/0.170/0.000 ms

2019-05-15T16:12:01.543Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:12:01.666Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ cut -d / -f 1
++ awk '{print $2}'
++ grep 'inet '
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.183 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.183/0.183/0.183/0.000 ms

2019-05-15T16:12:02.667Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:12:02.811Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ grep 'inet '
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.173 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.173/0.173/0.173/0.000 ms

2019-05-15T16:12:03.811Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:12:03.937Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.198 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.198/0.198/0.198/0.000 ms

2019-05-15T16:12:04.938Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh ping agent-1-vm1 agent-2-vm1 -c 1]
2019-05-15T16:12:05.081Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' ping == start ']'
+ '[' ping == stop ']'
+ '[' ping == stop-analyzer ']'
+ '[' ping == stop-agent ']'
+ '[' ping == ping ']'
+ '[' -z agent-2-vm1 ']'
+ ping agent-1-vm1 agent-2-vm1 -c 1
+ SRC=agent-1-vm1
+ DST=agent-2-vm1
+ ARGS='-c 1'
++ sudo ip netns exec agent-2-vm1 ip a show eth0
++ grep 'inet '
++ awk '{print $2}'
++ cut -d / -f 1
+ IP_DST=10.0.2.1
+ sudo ip netns exec agent-1-vm1 ping 10.0.2.1 -c 1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.244 ms

--- 10.0.2.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.244/0.244/0.244/0.000 ms

2019-05-15T16:12:06.527Z	DEBUG	tests/tests.go:305 execCmds	jenkins-slave-2.rdocloud: Executing command [/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive/scripts/scale.sh stop 2 4 2]
2019-05-15T16:12:14.960Z	DEBUG	tests/tests.go:308 execCmds	jenkins-slave-2.rdocloud: Command returned + check_dependencies
++ which inotifywait
+ '[' '!' /usr/bin/inotifywait ']'
++ which realpath
+ '[' '!' /usr/bin/realpath ']'
+ '[' stop == start ']'
+ '[' stop == stop ']'
+ '[' -z 2 ']'
+ stop 2 4 2
+ ANALYZER_NUM=2
+ AGENT_NUM=4
+ VM_NUM=2
+ echo 'Stopping Agents...'
Stopping Agents...
++ seq 4
+ for A in $( seq $AGENT_NUM )
+ stop_agent 1
+ AGENT=1
+ NAME=agent-1
+ '[' -e /tmp/skydive-scale/agent-1.pid ']'
+ sudo pkill -f agent-1.yml
+ timeout=30
+ pkill -0 -f agent-1.yml
+ sudo rm /tmp/skydive-scale/agent-1.lock /tmp/skydive-scale/agent-1.pid
rm: cannot remove '/tmp/skydive-scale/agent-1.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 2
+ AGENT=2
+ NAME=agent-2
+ '[' -e /tmp/skydive-scale/agent-2.pid ']'
+ sudo pkill -f agent-2.yml
+ timeout=30
+ pkill -0 -f agent-2.yml
+ sudo rm /tmp/skydive-scale/agent-2.lock /tmp/skydive-scale/agent-2.pid
rm: cannot remove '/tmp/skydive-scale/agent-2.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 3
+ AGENT=3
+ NAME=agent-3
+ '[' -e /tmp/skydive-scale/agent-3.pid ']'
+ sudo pkill -f agent-3.yml
+ timeout=30
+ pkill -0 -f agent-3.yml
+ sudo rm /tmp/skydive-scale/agent-3.lock /tmp/skydive-scale/agent-3.pid
rm: cannot remove '/tmp/skydive-scale/agent-3.pid': No such file or directory
+ for A in $( seq $AGENT_NUM )
+ stop_agent 4
+ AGENT=4
+ NAME=agent-4
+ '[' -e /tmp/skydive-scale/agent-4.pid ']'
+ sudo pkill -f agent-4.yml
+ timeout=30
+ pkill -0 -f agent-4.yml
+ sudo rm /tmp/skydive-scale/agent-4.lock /tmp/skydive-scale/agent-4.pid
rm: cannot remove '/tmp/skydive-scale/agent-4.lock': No such file or directory
rm: cannot remove '/tmp/skydive-scale/agent-4.pid': No such file or directory
+ echo 'Stopping Analyzers...'
Stopping Analyzers...
++ seq 2
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 1
+ ANALYZER=1
+ NAME=analyzer-1
+ '[' -e /tmp/skydive-scale/analyzer-1.pid ']'
+ sudo pkill -f analyzer-1.yml
+ timeout=30
+ pkill -0 -f analyzer-1.yml
+ sudo rm /tmp/skydive-scale/analyzer-1.lock /tmp/skydive-scale/analyzer-1.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-1.pid': No such file or directory
+ for A in $( seq $ANALYZER_NUM )
+ stop_analyzer 2
+ ANALYZER=2
+ NAME=analyzer-2
+ '[' -e /tmp/skydive-scale/analyzer-2.pid ']'
+ sudo pkill -f analyzer-2.yml
+ timeout=30
+ pkill -0 -f analyzer-2.yml
+ sudo rm /tmp/skydive-scale/analyzer-2.lock /tmp/skydive-scale/analyzer-2.pid
rm: cannot remove '/tmp/skydive-scale/analyzer-2.pid': No such file or directory
+ sleep 5
+ sudo -E screen -S skydive-stress -X quit
+ rm /tmp/skydive-scale/screen.lock
++ seq 4
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 1 2
+ IDX=1
+ VM_NUM=2
+ NAME=agent-1
+ echo Delete agent agent-1
Delete agent agent-1
++ cat /tmp/skydive-scale/agent-1-vswitchd.pid
+ sudo kill -9 19737
++ cat /tmp/skydive-scale/agent-1-dbserver.pid
+ sudo kill -9 19730
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm1
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-1-vm2
+ rm /tmp/skydive-scale/agent-1.lock
rm: cannot remove '/tmp/skydive-scale/agent-1.lock': No such file or directory
+ delete_host agent-1
+ NAME=agent-1
+ echo Delete Host agent-1
Delete Host agent-1
+ sudo ovs-vsctl del-port br-central agent-1-eth0
ovs-vsctl: no port named agent-1-eth0
+ sudo ip link del agent-1-eth0
+ sudo ip netns del agent-1
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 2 2
+ IDX=2
+ VM_NUM=2
+ NAME=agent-2
+ echo Delete agent agent-2
Delete agent agent-2
++ cat /tmp/skydive-scale/agent-2-vswitchd.pid
+ sudo kill -9 19935
++ cat /tmp/skydive-scale/agent-2-dbserver.pid
+ sudo kill -9 19929
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm1
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-2-vm2
+ rm /tmp/skydive-scale/agent-2.lock
rm: cannot remove '/tmp/skydive-scale/agent-2.lock': No such file or directory
+ delete_host agent-2
+ NAME=agent-2
+ echo Delete Host agent-2
Delete Host agent-2
+ sudo ovs-vsctl del-port br-central agent-2-eth0
ovs-vsctl: no port named agent-2-eth0
+ sudo ip link del agent-2-eth0
+ sudo ip netns del agent-2
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 3 2
+ IDX=3
+ VM_NUM=2
+ NAME=agent-3
+ echo Delete agent agent-3
Delete agent agent-3
++ cat /tmp/skydive-scale/agent-3-vswitchd.pid
+ sudo kill -9 20700
++ cat /tmp/skydive-scale/agent-3-dbserver.pid
+ sudo kill -9 20695
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm1
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-3-vm2
+ rm /tmp/skydive-scale/agent-3.lock
rm: cannot remove '/tmp/skydive-scale/agent-3.lock': No such file or directory
+ delete_host agent-3
+ NAME=agent-3
+ echo Delete Host agent-3
Delete Host agent-3
+ sudo ovs-vsctl del-port br-central agent-3-eth0
ovs-vsctl: no port named agent-3-eth0
+ sudo ip link del agent-3-eth0
+ sudo ip netns del agent-3
+ for AGENT_I in $( seq $AGENT_NUM )
+ delete_agent 4 2
+ IDX=4
+ VM_NUM=2
+ NAME=agent-4
+ echo Delete agent agent-4
Delete agent agent-4
++ cat /tmp/skydive-scale/agent-4-vswitchd.pid
+ sudo kill -9 20888
++ cat /tmp/skydive-scale/agent-4-dbserver.pid
+ sudo kill -9 20883
++ seq 2
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm1
+ for VM_I in $( seq $VM_NUM )
+ sudo ip netns del agent-4-vm2
+ rm /tmp/skydive-scale/agent-4.lock
rm: cannot remove '/tmp/skydive-scale/agent-4.lock': No such file or directory
+ delete_host agent-4
+ NAME=agent-4
+ echo Delete Host agent-4
Delete Host agent-4
+ sudo ovs-vsctl del-port br-central agent-4-eth0
ovs-vsctl: no port named agent-4-eth0
+ sudo ip link del agent-4-eth0
+ sudo ip netns del agent-4
+ delete_main_bridge
+ echo Delete central bridge
Delete central bridge
+ sudo ip link del br-central
+ sudo find /tmp/skydive-scale -mindepth 1 '!' -name '*.log' -a '!' -name '*.cover' -exec rm -rf '{}' ';'
find: ‘/tmp/skydive-scale/agent-4-netns’: No such file or directory
find: ‘/tmp/skydive-scale/agent-1-netns’: No such file or directory
find: ‘/tmp/skydive-scale/analyzer-2-etcd’: No such file or directory
find: ‘/tmp/skydive-scale/agent-3-netns’: No such file or directory
find: ‘/tmp/skydive-scale/agent-2-netns’: No such file or directory
find: ‘/tmp/skydive-scale/analyzer-1-etcd’: No such file or directory

--- PASS: TestScaleHA (213.62s)
PASS
make[1]: Leaving directory '/var/lib/jenkins/workspace/skydive-scale-tests/src/github.com/skydive-project/skydive'
status=$?

cat /tmp/skydive-scale/{analyzer,agent}-?.log | sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" | perl -ne '$d=$1 if /^(\d+-\d+-\d+),/; $k{$d}.=$_; END{print $k{$_} for sort keys(%k);}'
2019-05-15T16:08:43.331Z	INFO	analyzer/analyzer.go:42 glob..func1	analyzer-1: Skydive Analyzer 0.22.0-5bebd28d8830 starting...
2019-05-15 16:08:43.332393 I | embed: listening for peers on http://192.168.50.254:12380
2019-05-15 16:08:43.332531 I | embed: listening for client requests on 192.168.50.254:12379
2019-05-15 16:08:43.336547 I | etcdserver: name = jenkins-slave-2.rdocloud
2019-05-15 16:08:43.336573 I | etcdserver: data dir = /tmp/skydive-scale/analyzer-1-etcd
2019-05-15 16:08:43.336582 I | etcdserver: member dir = /tmp/skydive-scale/analyzer-1-etcd/member
2019-05-15 16:08:43.336594 I | etcdserver: heartbeat = 100ms
2019-05-15 16:08:43.336600 I | etcdserver: election = 1000ms
2019-05-15 16:08:43.336691 I | etcdserver: snapshot count = 100000
2019-05-15 16:08:43.336718 I | etcdserver: advertise client URLs = http://192.168.50.254:12379
2019-05-15 16:08:43.336783 I | etcdserver: initial advertise peer URLs = http://localhost:12380
2019-05-15 16:08:43.336810 I | etcdserver: initial cluster = jenkins-slave-2.rdocloud=http://localhost:12380
2019-05-15 16:08:43.348353 I | etcdserver: starting member c9ac9fc89eae9cf7 in cluster f290f7f84a022555
2019-05-15 16:08:43.348428 I | raft: c9ac9fc89eae9cf7 became follower at term 0
2019-05-15 16:08:43.348457 I | raft: newRaft c9ac9fc89eae9cf7 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2019-05-15 16:08:43.348469 I | raft: c9ac9fc89eae9cf7 became follower at term 1
2019-05-15 16:08:43.358446 W | auth: simple token is not cryptographically signed
2019-05-15 16:08:43.359766 I | etcdserver: starting server... [version: 3.2.15+git, cluster version: to_be_decided]
2019-05-15 16:08:43.360378 I | etcdserver/membership: added member c9ac9fc89eae9cf7 [http://localhost:12380] to cluster f290f7f84a022555
2019-05-15 16:08:43.850036 I | raft: c9ac9fc89eae9cf7 is starting a new election at term 1
2019-05-15 16:08:43.850199 I | raft: c9ac9fc89eae9cf7 became candidate at term 2
2019-05-15 16:08:43.850231 I | raft: c9ac9fc89eae9cf7 received MsgVoteResp from c9ac9fc89eae9cf7 at term 2
2019-05-15 16:08:43.850258 I | raft: c9ac9fc89eae9cf7 became leader at term 2
2019-05-15 16:08:43.850268 I | raft: raft.node: c9ac9fc89eae9cf7 elected leader c9ac9fc89eae9cf7 at term 2
2019-05-15 16:08:43.850702 I | etcdserver: published {Name:jenkins-slave-2.rdocloud ClientURLs:[http://192.168.50.254:12379]} to cluster f290f7f84a022555
2019-05-15 16:08:43.850722 I | Server is ready!
2019-05-15 16:08:43.851406 I | etcdserver: setting up the initial cluster version to 3.2
2019-05-15 16:08:43.851470 I | embed: ready to serve client requests
2019-05-15 16:08:43.851766 N | embed: serving insecure client requests on 192.168.50.254:12379, this is strongly discouraged!
2019-05-15 16:08:43.853063 N | etcdserver/membership: set the initial cluster version to 3.2
2019-05-15 16:08:43.853131 I | etcdserver/api: enabled capabilities for version 3.2
2019-05-15T16:08:43.853Z	DEBUG	etcd/server.go:136 NewEmbeddedEtcd	analyzer-1: Successfully started etcd
2019-05-15T16:08:43.854Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Model:
2019-05-15T16:08:43.854Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Model:
2019-05-15T16:08:43.854Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: r.r: sub, obj, act
2019-05-15T16:08:43.854Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: p.p: sub, obj, act, eft
2019-05-15T16:08:43.854Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: e.e: some(where (p_eft == allow)) && !some(where (p_eft == deny))
2019-05-15T16:08:43.854Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: m.m: g(r_sub, p_sub) && r_obj == p_obj && r_act == p_act
2019-05-15T16:08:43.854Z	DEBUG	config/rbac.go:53 (*logger).Printf	analyzer-1: g.g: _, _
2019-05-15T16:08:43.857Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:08:43.857Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: 
2019-05-15T16:08:43.859Z	INFO	analyzer/storage.go:59 newGraphBackendFromConfig	analyzer-1: Using elasticsearch (driver elasticsearch) as graph storage backend
2019-05-15T16:08:43.860Z	INFO	etcd/election.go:115 (*MasterElector).start	analyzer-1: starting as a follower for /master-analyzer-es-graph-flush: analyzer-1
2019-05-15T16:08:45.314Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-es-rolling-index:1d8779e5: analyzer-1
2019-05-15T16:08:45.314Z	INFO	elasticsearch/client.go:225 (*Client).start	analyzer-1: client started for skydive_topology_live, skydive_topology_archive
2019-05-15T16:08:45.316Z	INFO	analyzer/storage.go:85 newFlowBackendFromConfig	analyzer-1: Using elasticsearch (driver elasticsearch) as flow storage backend
2019-05-15T16:08:45.399Z	INFO	http/server.go:104 (*Server).Listen	analyzer-1: Listening on socket 0.0.0.0:8082
2019-05-15T16:08:45.400Z	INFO	hub/replication_endpoint.go:140 (*ReplicatorPeer).connect	analyzer-1: Connecting to peer: wss://[::1]:8082/ws/replication
2019-05-15T16:08:45.400Z	INFO	hub/replication_endpoint.go:140 (*ReplicatorPeer).connect	analyzer-1: Connecting to peer: wss://[::1]:8084/ws/replication
2019-05-15T16:08:45.401Z	INFO	websocket/client.go:495 (*Client).Connect	analyzer-1: Connecting to wss://[::1]:8084/ws/replication
2019-05-15T16:08:45.401Z	INFO	websocket/client.go:495 (*Client).Connect	analyzer-1: Connecting to wss://[::1]:8082/ws/replication
2019-05-15T16:08:45.402Z	ERROR	websocket/client.go:557 (*Client).Start.func1	analyzer-1: Unable to create a WebSocket connection wss://[::1]:8084/ws/replication : dial tcp [::1]:8084: connect: connection refused
2019-05-15T16:08:45.403Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-ondemand-client: analyzer-1
2019-05-15T16:08:45.406Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-pi-client: analyzer-1
2019-05-15T16:08:45.417Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-alert-server: analyzer-1
2019-05-15T16:08:45.421Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-topology-manager: analyzer-1
2019-05-15T16:08:45.423Z	INFO	analyzer/analyzer.go:55 glob..func1	analyzer-1: Skydive Analyzer started !
2019-05-15T16:08:45.557Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:08:45.557Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: analyzer-1 < admin
2019-05-15T16:08:45.558Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/replication, analyzer-1
2019-05-15T16:08:45.565Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: analyzer-1, websocket, /ws/replication ---> true
2019-05-15T16:08:45.565Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client analyzer-1 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:08:45.565Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from [::1]:48740 : URI path /ws/replication
2019-05-15T16:08:45.565Z	INFO	websocket/client.go:519 (*Client).Connect	analyzer-1: Connected to wss://[::1]:8082/ws/replication
2019-05-15T16:08:45.565Z	DEBUG	hub/replication_endpoint.go:88 (*ReplicatorPeer).OnConnected	analyzer-1: Disconnecting from wss://[::1]:8082/ws/replication since it's me
2019-05-15T16:08:45.565Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient analyzer-1 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:08:45.566Z	DEBUG	websocket/pool.go:85 (*Pool).OnDisconnected	analyzer-1: OnDisconnected analyzer-1 for pool /ws/replication type : [*websocket.Pool] 
2019-05-15T16:08:45.566Z	DEBUG	websocket/pool.go:143 (*Pool).RemoveClient	analyzer-1: Successfully removed client analyzer-1 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:08:46.075Z	INFO	etcd/election.go:103 (*MasterElector).start	analyzer-1: starting as the master for /master-analyzer-es-rolling-index:3f0e228a: analyzer-1
2019-05-15T16:08:46.075Z	INFO	elasticsearch/client.go:225 (*Client).start	analyzer-1: client started for skydive_flow, skydive_metric, skydive_rawpacket
2019-05-15T16:08:46.311Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:08:46.311Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: analyzer-1 < admin, analyzer-2 < admin
2019-05-15T16:08:46.312Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/replication, analyzer-2
2019-05-15T16:08:46.312Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: analyzer-2, websocket, /ws/replication ---> true
2019-05-15T16:08:46.312Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client analyzer-2 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:08:46.313Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from [::1]:48764 : URI path /ws/replication
2019-05-15T16:08:46.313Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient analyzer-2 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:08:46.402Z	INFO	websocket/client.go:495 (*Client).Connect	analyzer-1: Connecting to wss://[::1]:8084/ws/replication
2019-05-15T16:08:46.471Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:08:46.471Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: analyzer-1 < admin, analyzer-2 < admin, agent-1 < admin
2019-05-15T16:08:46.472Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/topology, agent-1
2019-05-15T16:08:46.473Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-1, websocket, /ws/agent/topology ---> true
2019-05-15T16:08:46.473Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-1 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:08:46.473Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.1:38726 : URI path /ws/agent/topology
2019-05-15T16:08:46.473Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-1 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:08:46.479Z	DEBUG	hub/pod_endpoint.go:70 (*AgentEndpoint).OnStructMessage	analyzer-1: Authoritative client registered agent.agent-1
2019-05-15T16:08:46.540Z	INFO	websocket/client.go:519 (*Client).Connect	analyzer-1: Connected to wss://[::1]:8084/ws/replication
2019-05-15T16:08:46.542Z	DEBUG	hub/replication_endpoint.go:95 (*ReplicatorPeer).OnConnected	analyzer-1: Disconnecting from wss://[::1]:8084/ws/replication as already connected through wss://[::1]:8084/ws/replication
2019-05-15T16:08:46.601Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/flow, agent-1
2019-05-15T16:08:46.601Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-1, websocket, /ws/agent/flow ---> true
2019-05-15T16:08:46.601Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-1 for pool /ws/agent/flow type : [*websocket.Pool]
2019-05-15T16:08:46.602Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.1:38732 : URI path /ws/agent/flow
2019-05-15T16:08:46.602Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-1 for pool /ws/agent/flow type : [*websocket.Pool]
2019-05-15T16:08:47.556Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:08:47.556Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: agent-1 < admin, agent-2 < admin, analyzer-1 < admin, analyzer-2 < admin
2019-05-15T16:08:47.558Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/topology, agent-2
2019-05-15T16:08:47.558Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-2, websocket, /ws/agent/topology ---> true
2019-05-15T16:08:47.558Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-2 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:08:47.559Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.2:44234 : URI path /ws/agent/topology
2019-05-15T16:08:47.559Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-2 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:08:47.577Z	DEBUG	hub/pod_endpoint.go:70 (*AgentEndpoint).OnStructMessage	analyzer-1: Authoritative client registered agent.agent-2
2019-05-15T16:08:47.667Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/flow, agent-2
2019-05-15T16:08:47.668Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-2, websocket, /ws/agent/flow ---> true
2019-05-15T16:08:47.668Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-2 for pool /ws/agent/flow type : [*websocket.Pool]
2019-05-15T16:08:47.668Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.2:44238 : URI path /ws/agent/flow
2019-05-15T16:08:47.668Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-2 for pool /ws/agent/flow type : [*websocket.Pool]
2019-05-15T16:09:18.279Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:09:18.279Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: agent-2 < admin, analyzer-1 < admin, admin < admin, analyzer-2 < admin, agent-1 < admin
2019-05-15T16:09:18.281Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:09:18.378Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:09:18.382Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, capture, write ---> true
2019-05-15T16:09:18.385Z	DEBUG	client/client.go:357 (*OnDemandProbeClient).onAPIWatcherEvent	analyzer-1: New watcher event set for 4fc1e347-59f5-4319-4768-9e4d39ac6c29
2019-05-15T16:09:18.395Z	DEBUG	client/client.go:83 (*OnDemandProbeClient).OnStructMessage	analyzer-1: Capture start request succeeded Namespace OnDemand Type CaptureStartReply UUID 1bbc07a7-6255-40be-75e1-b666cb815398 Status 200 Obj (347 bytes)
2019-05-15T16:09:18.397Z	DEBUG	client/client.go:83 (*OnDemandProbeClient).OnStructMessage	analyzer-1: Capture start request succeeded Namespace OnDemand Type CaptureStartReply UUID e69436bb-bd30-4da4-7eb9-3e6cf308c4b3 Status 200 Obj (347 bytes)
2019-05-15T16:09:23.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:d2053a9334db7a3f LayersPath:Ethernet/ARP Application:ARP Link:A:"e2:61:85:03:4b:2c" B:"ff:ff:ff:ff:ff:ff"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936558631 Last:1557936563391  Metric:ABPackets:1 ABBytes:42 Start:1557936558631 Last:1557936558631  TCPMetric:<nil> IPMetric:<nil> Start:1557936558631 Last:1557936558631 TrackingID:6669fdb0e022a229 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:23.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:5fbc738c487ffa4e LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"e2:61:85:03:4b:2c" B:"c6:dd:cf:20:29:0e"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:20114  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936558632 Last:1557936563391 RTT:36000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936558632 Last:1557936558632 RTT:36000  TCPMetric:<nil> IPMetric: Start:1557936558632 Last:1557936558632 TrackingID:e076d8710fb07157 L3TrackingID:78b298e6f6c5b263 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:23.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:825fe1339c3b6ec2 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"4e:5d:de:31:7f:d9" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::4c5d:deff:fe31:7fd9" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936560191 Last:1557936563391  Metric:ABPackets:1 ABBytes:70 Start:1557936560191 Last:1557936560191  TCPMetric:<nil> IPMetric:<nil> Start:1557936560191 Last:1557936560191 TrackingID:4bd73e2932986676 L3TrackingID:4c3472d36eb478c6 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:23.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:d178b2535470d223 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"56:45:6e:08:6a:94" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::5445:6eff:fe08:6a94" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936560191 Last:1557936563391  Metric:ABPackets:1 ABBytes:70 Start:1557936560191 Last:1557936560191  TCPMetric:<nil> IPMetric:<nil> Start:1557936560191 Last:1557936560191 TrackingID:2741bdbb5b634308 L3TrackingID:132fd51018ac73bd ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:23.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:dac2e05e1faec18f LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"e2:61:85:03:4b:2c" B:"c6:dd:cf:20:29:0e"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:20150  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936560917 Last:1557936563391 RTT:22000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936560917 Last:1557936560917 RTT:22000  TCPMetric:<nil> IPMetric: Start:1557936560917 Last:1557936560917 TrackingID:9040c41919f48596 L3TrackingID:995632f5b0bdfc8c ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:23.394Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:d4000a50882e5a0d LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"e2:61:85:03:4b:2c" B:"c6:dd:cf:20:29:0e"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:20166  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936562031 Last:1557936563391 RTT:15000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936562031 Last:1557936562031 RTT:15000  TCPMetric:<nil> IPMetric: Start:1557936562031 Last:1557936562031 TrackingID:3ab965fcec82ec6b L3TrackingID:86cd6301fd40d80d ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:23.394Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:b12807c49c413d4c LayersPath:Ethernet/ARP Application:ARP Link:A:"c6:dd:cf:20:29:0e" B:"e2:61:85:03:4b:2c"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936558631 Last:1557936563391  Metric:ABPackets:1 ABBytes:42 Start:1557936558631 Last:1557936558631  TCPMetric:<nil> IPMetric:<nil> Start:1557936558631 Last:1557936558631 TrackingID:eea1c9c27959951d L3TrackingID:5752b0859882550f ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:23.394Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e19953540b36f848 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"4e:5d:de:31:7f:d9" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936559227 Last:1557936563391  Metric:ABPackets:2 ABBytes:106 Start:1557936559227 Last:1557936561231  TCPMetric:<nil> IPMetric:<nil> Start:1557936559227 Last:1557936561231 TrackingID:95a343d976ba3ebe L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:23.394Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:91a2861b1425ed7e LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"e2:61:85:03:4b:2c" B:"c6:dd:cf:20:29:0e"  Network:Protocol:IPV4 A:"10.0.1.1" B:"10.0.2.1"  Transport:<nil> ICMP:Type:ECHO ID:20130  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936559793 Last:1557936563391 RTT:24000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936559793 Last:1557936559793 RTT:24000  TCPMetric:<nil> IPMetric: Start:1557936559793 Last:1557936559793 TrackingID:dcaca3d5a0a5a546 L3TrackingID:cc432793ab1916df ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:25.424Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 9 flows stored
2019-05-15T16:09:45.315Z	DEBUG	elasticsearch/rollindex.go:83 (*rollIndexService).roll	analyzer-1: Start rolling indices...
2019-05-15T16:09:45.315Z	INFO	elasticsearch/rollindex.go:120 (*rollIndexService).roll	analyzer-1: Rolling indices done
2019-05-15T16:09:46.075Z	DEBUG	elasticsearch/rollindex.go:83 (*rollIndexService).roll	analyzer-1: Start rolling indices...
2019-05-15T16:09:46.075Z	INFO	elasticsearch/rollindex.go:120 (*rollIndexService).roll	analyzer-1: Rolling indices done
2019-05-15T16:09:58.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:bdede9b624288b7b LayersPath:Ethernet/ARP Application:ARP Link:A:"e2:61:85:03:4b:2c" B:"c6:dd:cf:20:29:0e"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:Start:1557936588391 Last:1557936593391  Metric:ABPackets:2 ABBytes:84 BAPackets:2 BABytes:84 Start:1557936582208 Last:1557936587843  TCPMetric:<nil> IPMetric:<nil> Start:1557936582208 Last:1557936587843 TrackingID:eea1c9c27959951d L3TrackingID:5752b0859882550f ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:TIMEOUT}
2019-05-15T16:09:58.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e19953540b36f848 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"4e:5d:de:31:7f:d9" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936593391 Last:1557936598391  Metric:ABPackets:19 ABBytes:1007 Start:1557936559227 Last:1557936596273  TCPMetric:<nil> IPMetric:<nil> Start:1557936559227 Last:1557936596273 TrackingID:95a343d976ba3ebe L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:58.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:d178b2535470d223 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"56:45:6e:08:6a:94" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::5445:6eff:fe08:6a94" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936593391 Last:1557936598391  Metric:ABPackets:2 ABBytes:140 Start:1557936560191 Last:1557936593471  TCPMetric:<nil> IPMetric:<nil> Start:1557936560191 Last:1557936593471 TrackingID:2741bdbb5b634308 L3TrackingID:132fd51018ac73bd ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:58.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:39cb83f158ebb5a7 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"56:45:6e:08:6a:94" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::5445:6eff:fe08:6a94" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936593393 Last:1557936598393  Metric:ABPackets:2 ABBytes:140 Start:1557936560191 Last:1557936593471  TCPMetric:<nil> IPMetric:<nil> Start:1557936560191 Last:1557936593471 TrackingID:2741bdbb5b634308 L3TrackingID:132fd51018ac73bd ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:09:58.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:c3885be5c8c7cbf0 LayersPath:Ethernet/ARP Application:ARP Link:A:"e2:61:85:03:4b:2c" B:"c6:dd:cf:20:29:0e"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:Start:1557936588393 Last:1557936593393  Metric:ABPackets:2 ABBytes:84 BAPackets:2 BABytes:84 Start:1557936582207 Last:1557936587843  TCPMetric:<nil> IPMetric:<nil> Start:1557936582207 Last:1557936587843 TrackingID:eea1c9c27959951d L3TrackingID:5752b0859882550f ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:TIMEOUT}
2019-05-15T16:09:58.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:eb00921c15bc9380 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"66:3b:f6:27:8c:8f" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936593393 Last:1557936598393  Metric:ABPackets:19 ABBytes:1007 Start:1557936559707 Last:1557936597246  TCPMetric:<nil> IPMetric:<nil> Start:1557936559707 Last:1557936597246 TrackingID:7cfa23dc758b6d05 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:00.424Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 6 flows stored
2019-05-15T16:10:05.591Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:10:05.591Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: analyzer-1 < admin, admin < admin, analyzer-2 < admin, agent-1 < admin, agent-2 < admin, agent-3 < admin
2019-05-15T16:10:05.599Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/topology, agent-3
2019-05-15T16:10:05.599Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-3, websocket, /ws/agent/topology ---> true
2019-05-15T16:10:05.599Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-3 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:10:05.606Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.3:58920 : URI path /ws/agent/topology
2019-05-15T16:10:05.606Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-3 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:10:05.607Z	DEBUG	hub/pod_endpoint.go:70 (*AgentEndpoint).OnStructMessage	analyzer-1: Authoritative client registered agent.agent-3
2019-05-15T16:10:05.764Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/flow, agent-3
2019-05-15T16:10:05.764Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-3, websocket, /ws/agent/flow ---> true
2019-05-15T16:10:05.764Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-3 for pool /ws/agent/flow type : [*websocket.Pool]
2019-05-15T16:10:05.764Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.3:58924 : URI path /ws/agent/flow
2019-05-15T16:10:05.765Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-3 for pool /ws/agent/flow type : [*websocket.Pool]
2019-05-15T16:10:06.327Z	DEBUG	client/client.go:83 (*OnDemandProbeClient).OnStructMessage	analyzer-1: Capture start request succeeded Namespace OnDemand Type CaptureStartReply UUID 6453a67c-a78d-4755-443a-6d4cb3234694 Status 200 Obj (347 bytes)
2019-05-15T16:10:07.926Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Role links for: g
2019-05-15T16:10:07.926Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: agent-2 < admin, agent-3 < admin, agent-4 < admin, analyzer-1 < admin, admin < admin, analyzer-2 < admin, agent-1 < admin
2019-05-15T16:10:07.934Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/topology, agent-4
2019-05-15T16:10:07.934Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-4, websocket, /ws/agent/topology ---> true
2019-05-15T16:10:07.934Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-4 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:10:07.934Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.4:55246 : URI path /ws/agent/topology
2019-05-15T16:10:07.935Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-4 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:10:08.070Z	DEBUG	websocket/server.go:67 (*Server).serveMessages	analyzer-1: Enforcing websocket for /ws/agent/flow, agent-4
2019-05-15T16:10:08.070Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: agent-4, websocket, /ws/agent/flow ---> true
2019-05-15T16:10:08.070Z	DEBUG	websocket/server.go:79 (*Server).serveMessages	analyzer-1: Serving messages for client agent-4 for pool /ws/agent/flow type : [*websocket.Pool]
2019-05-15T16:10:08.070Z	INFO	websocket/server.go:122 (*Server).newIncomingClient	analyzer-1: New WebSocket Connection from 192.168.50.4:55252 : URI path /ws/agent/flow
2019-05-15T16:10:08.071Z	DEBUG	websocket/pool.go:115 (*incomerPool).AddClient	analyzer-1: AddClient agent-4 for pool /ws/agent/flow type : [*websocket.Pool]
2019-05-15T16:10:08.327Z	DEBUG	client/client.go:83 (*OnDemandProbeClient).OnStructMessage	analyzer-1: Capture start request succeeded Namespace OnDemand Type CaptureStartReply UUID bd295d53-58e1-461b-7fdb-19a09806207c Status 200 Obj (347 bytes)
2019-05-15T16:10:08.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e19953540b36f848 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"4e:5d:de:31:7f:d9" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936603391 Last:1557936608391  Metric:ABPackets:24 ABBytes:1272 Start:1557936559227 Last:1557936607283  TCPMetric:<nil> IPMetric:<nil> Start:1557936559227 Last:1557936607283 TrackingID:95a343d976ba3ebe L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:08.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:eb00921c15bc9380 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"66:3b:f6:27:8c:8f" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936603393 Last:1557936608392  Metric:ABPackets:23 ABBytes:1219 Start:1557936559707 Last:1557936605264  TCPMetric:<nil> IPMetric:<nil> Start:1557936559707 Last:1557936605264 TrackingID:7cfa23dc758b6d05 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:09.929Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:10:10.291Z	DEBUG	websocket/pool.go:85 (*Pool).OnDisconnected	analyzer-1: OnDisconnected agent-4 for pool /ws/agent/flow type : [*websocket.Pool] 
2019-05-15T16:10:10.291Z	DEBUG	websocket/pool.go:143 (*Pool).RemoveClient	analyzer-1: Successfully removed client agent-4 for pool /ws/agent/flow type : [*websocket.Pool]
2019-05-15T16:10:10.292Z	DEBUG	websocket/pool.go:85 (*Pool).OnDisconnected	analyzer-1: OnDisconnected agent-4 for pool /ws/agent/topology type : [*websocket.Pool] 
2019-05-15T16:10:10.292Z	DEBUG	websocket/pool.go:143 (*Pool).RemoveClient	analyzer-1: Successfully removed client agent-4 for pool /ws/agent/topology type : [*websocket.Pool]
2019-05-15T16:10:10.426Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 2 flows stored
2019-05-15T16:10:10.471Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:10:10.598Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, topology, read ---> true
2019-05-15T16:10:10.750Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:10:10.980Z	DEBUG	websocket/pool.go:85 (*Pool).OnDisconnected	analyzer-1: OnDisconnected analyzer-2 for pool /ws/replication type : [*websocket.Pool] 
2019-05-15T16:10:10.981Z	DEBUG	hub/replication_endpoint.go:387 (*ReplicationEndpoint).OnDisconnected	analyzer-1: Peer unregistered, delete resources of analyzer.analyzer-2
2019-05-15T16:10:10.981Z	DEBUG	websocket/pool.go:143 (*Pool).RemoveClient	analyzer-1: Successfully removed client analyzer-2 for pool /ws/replication type : [*websocket.Pool]
2019-05-15T16:10:11.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:862266a0c94c5a85 LayersPath:Ethernet/IPv6/IPv6HopByHop/ICMPv6/MLDv2MulticastListenerReport Application:MLDv2MulticastListenerReport Link:A:"ce:a9:cc:5a:72:fe" B:"33:33:00:00:00:16"  Network:Protocol:IPV6 A:"fe80::cca9:ccff:fe5a:72fe" B:"ff02::16"  Transport:<nil> ICMP: DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:180 Start:1557936606719 Last:1557936611325  Metric:ABPackets:2 ABBytes:180 Start:1557936606719 Last:1557936607679  TCPMetric:<nil> IPMetric:<nil> Start:1557936606719 Last:1557936607679 TrackingID:ef07ae93f32164ec L3TrackingID:08687fc97bf755e0 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:11.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:fbcb3453920415a0 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"ce:a9:cc:5a:72:fe" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::cca9:ccff:fe5a:72fe" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:140 Start:1557936606719 Last:1557936611325  Metric:ABPackets:2 ABBytes:140 Start:1557936606719 Last:1557936610430  TCPMetric:<nil> IPMetric:<nil> Start:1557936606719 Last:1557936610430 TrackingID:a1ee67a15987c4c0 L3TrackingID:66975eade8d9e1d2 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:11.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:2ead1bc380c33393 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"ae:fe:9b:c1:90:e2" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:4 ABBytes:212 Start:1557936606958 Last:1557936611325  Metric:ABPackets:4 ABBytes:212 Start:1557936606958 Last:1557936609741  TCPMetric:<nil> IPMetric:<nil> Start:1557936606958 Last:1557936609741 TrackingID:47b108140bcd1f6d L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:11.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:20ff557f2481caf2 LayersPath:Ethernet/IPv6/IPv6HopByHop/ICMPv6/MLDv2MulticastListenerReport Application:MLDv2MulticastListenerReport Link:A:"ae:fe:9b:c1:90:e2" B:"33:33:00:00:00:16"  Network:Protocol:IPV6 A:"fe80::acfe:9bff:fec1:90e2" B:"ff02::16"  Transport:<nil> ICMP: DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:90 Start:1557936607039 Last:1557936611325  Metric:ABPackets:1 ABBytes:90 Start:1557936607039 Last:1557936607039  TCPMetric:<nil> IPMetric:<nil> Start:1557936607039 Last:1557936607039 TrackingID:bb739f56d0778090 L3TrackingID:37d6f27635e89269 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:11.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:af4e60ee2ee60245 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"ae:fe:9b:c1:90:e2" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::acfe:9bff:fec1:90e2" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936610367 Last:1557936611325  Metric:ABPackets:1 ABBytes:70 Start:1557936610367 Last:1557936610367  TCPMetric:<nil> IPMetric:<nil> Start:1557936610367 Last:1557936610367 TrackingID:134e0993dc145d21 L3TrackingID:4d1954f153cda53e ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:13.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:0510c35fb4fa2fd9 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"5a:4a:c0:33:49:98" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::584a:c0ff:fe33:4998" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936611071 Last:1557936613391  Metric:ABPackets:1 ABBytes:70 Start:1557936611071 Last:1557936611071  TCPMetric:<nil> IPMetric:<nil> Start:1557936611071 Last:1557936611071 TrackingID:3128f626dec4d6f8 L3TrackingID:e173bd00964a8d95 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:13.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e19953540b36f848 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"4e:5d:de:31:7f:d9" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936608391 Last:1557936613391  Metric:ABPackets:27 ABBytes:1431 Start:1557936559227 Last:1557936612256  TCPMetric:<nil> IPMetric:<nil> Start:1557936559227 Last:1557936612256 TrackingID:95a343d976ba3ebe L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:13.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:ef3c410a8b7d013b LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"ce:a9:cc:5a:72:fe" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::cca9:ccff:fe5a:72fe" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936610432 Last:1557936613391  Metric:ABPackets:1 ABBytes:70 Start:1557936610432 Last:1557936610432  TCPMetric:<nil> IPMetric:<nil> Start:1557936610432 Last:1557936610432 TrackingID:a1ee67a15987c4c0 L3TrackingID:66975eade8d9e1d2 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:13.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:24dc4d8dadd736c4 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"ce:a9:cc:5a:72:fe" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::cca9:ccff:fe5a:72fe" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936610432 Last:1557936613393  Metric:ABPackets:1 ABBytes:70 Start:1557936610432 Last:1557936610432  TCPMetric:<nil> IPMetric:<nil> Start:1557936610432 Last:1557936610432 TrackingID:a1ee67a15987c4c0 L3TrackingID:66975eade8d9e1d2 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:13.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:6f4852b8150c70ee LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"5a:4a:c0:33:49:98" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::584a:c0ff:fe33:4998" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936611071 Last:1557936613393  Metric:ABPackets:1 ABBytes:70 Start:1557936611071 Last:1557936611071  TCPMetric:<nil> IPMetric:<nil> Start:1557936611071 Last:1557936611071 TrackingID:3128f626dec4d6f8 L3TrackingID:e173bd00964a8d95 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:13.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:eb00921c15bc9380 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"66:3b:f6:27:8c:8f" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936608392 Last:1557936613393  Metric:ABPackets:26 ABBytes:1378 Start:1557936559707 Last:1557936611754  TCPMetric:<nil> IPMetric:<nil> Start:1557936559707 Last:1557936611754 TrackingID:7cfa23dc758b6d05 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:15.430Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 11 flows stored
2019-05-15T16:10:16.090Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:10:16.173Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, topology, read ---> true
2019-05-15T16:10:16.187Z	DEBUG	config/rbac.go:49 (*logger).Print	analyzer-1: Request: admin, status, read ---> true
2019-05-15T16:10:16.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:84ccdc30c6266898 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"5a:4a:c0:33:49:98" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::584a:c0ff:fe33:4998" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936611071 Last:1557936616325  Metric:ABPackets:1 ABBytes:70 Start:1557936611071 Last:1557936611071  TCPMetric:<nil> IPMetric:<nil> Start:1557936611071 Last:1557936611071 TrackingID:3128f626dec4d6f8 L3TrackingID:e173bd00964a8d95 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:16.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:48b95e98a8092ac8 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"de:56:b2:0a:9a:a8" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::dc56:b2ff:fe0a:9aa8" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936613439 Last:1557936616325  Metric:ABPackets:1 ABBytes:70 Start:1557936613439 Last:1557936613439  TCPMetric:<nil> IPMetric:<nil> Start:1557936613439 Last:1557936613439 TrackingID:be3fd8da376a363d L3TrackingID:27a834cddd0e848f ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:16.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:bd7369fcd2a9fe53 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"a2:5f:54:48:f3:b1" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::a05f:54ff:fe48:f3b1" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936613952 Last:1557936616325  Metric:ABPackets:1 ABBytes:70 Start:1557936613952 Last:1557936613952  TCPMetric:<nil> IPMetric:<nil> Start:1557936613952 Last:1557936613952 TrackingID:00aaf279c65ca7a5 L3TrackingID:c411d6b13e33f207 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:16.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:2ead1bc380c33393 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"ae:fe:9b:c1:90:e2" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936611325 Last:1557936616325  Metric:ABPackets:7 ABBytes:371 Start:1557936606958 Last:1557936615273  TCPMetric:<nil> IPMetric:<nil> Start:1557936606958 Last:1557936615273 TrackingID:47b108140bcd1f6d L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e377a17e74e58ab2 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"a2:5f:54:48:f3:b1" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::a05f:54ff:fe48:f3b1" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936613952 Last:1557936618391  Metric:ABPackets:1 ABBytes:70 Start:1557936613952 Last:1557936613952  TCPMetric:<nil> IPMetric:<nil> Start:1557936613952 Last:1557936613952 TrackingID:00aaf279c65ca7a5 L3TrackingID:c411d6b13e33f207 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e19953540b36f848 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"4e:5d:de:31:7f:d9" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936613391 Last:1557936618391  Metric:ABPackets:29 ABBytes:1537 Start:1557936559227 Last:1557936616280  TCPMetric:<nil> IPMetric:<nil> Start:1557936559227 Last:1557936616280 TrackingID:95a343d976ba3ebe L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:634c84e4947fe4db LayersPath:Ethernet/ARP Application:ARP Link:A:"ce:a9:cc:5a:72:fe" B:"ff:ff:ff:ff:ff:ff"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936616312 Last:1557936618391  Metric:ABPackets:1 ABBytes:42 Start:1557936616312 Last:1557936616312  TCPMetric:<nil> IPMetric:<nil> Start:1557936616312 Last:1557936616312 TrackingID:65001b5d84e20089 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:26ba0e3b1f16da74 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"de:56:b2:0a:9a:a8" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::dc56:b2ff:fe0a:9aa8" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936613439 Last:1557936618391  Metric:ABPackets:1 ABBytes:70 Start:1557936613439 Last:1557936613439  TCPMetric:<nil> IPMetric:<nil> Start:1557936613439 Last:1557936613439 TrackingID:be3fd8da376a363d L3TrackingID:27a834cddd0e848f ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:eb00921c15bc9380 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"66:3b:f6:27:8c:8f" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936613393 Last:1557936618393  Metric:ABPackets:29 ABBytes:1537 Start:1557936559707 Last:1557936617779  TCPMetric:<nil> IPMetric:<nil> Start:1557936559707 Last:1557936617779 TrackingID:7cfa23dc758b6d05 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:1430c78a2a0a0ade LayersPath:Ethernet/ARP Application:ARP Link:A:"ce:a9:cc:5a:72:fe" B:"ff:ff:ff:ff:ff:ff"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936616312 Last:1557936618393  Metric:ABPackets:1 ABBytes:42 Start:1557936616312 Last:1557936616312  TCPMetric:<nil> IPMetric:<nil> Start:1557936616312 Last:1557936616312 TrackingID:65001b5d84e20089 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:393d9d80fb7c1991 LayersPath:Ethernet/ARP Application:ARP Link:A:"e2:61:85:03:4b:2c" B:"ce:a9:cc:5a:72:fe"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936616312 Last:1557936618393  Metric:ABPackets:1 ABBytes:42 Start:1557936616312 Last:1557936616312  TCPMetric:<nil> IPMetric:<nil> Start:1557936616312 Last:1557936616312 TrackingID:b15487ce81f032e6 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:c63e728c68d12586 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"ce:a9:cc:5a:72:fe" B:"e2:61:85:03:4b:2c"  Network:Protocol:IPV4 A:"10.0.3.1" B:"10.0.1.1"  Transport:<nil> ICMP:Type:ECHO ID:21135  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936616313 Last:1557936618393 RTT:286000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936616313 Last:1557936616313 RTT:286000  TCPMetric:<nil> IPMetric: Start:1557936616313 Last:1557936616313 TrackingID:e61b81694ed446ab L3TrackingID:c746ce95f8c08654 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.394Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:2be674b9e4ea6fee LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"ce:a9:cc:5a:72:fe" B:"e2:61:85:03:4b:2c"  Network:Protocol:IPV4 A:"10.0.3.1" B:"10.0.1.1"  Transport:<nil> ICMP:Type:ECHO ID:21152  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936617439 Last:1557936618393 RTT:28000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936617439 Last:1557936617439 RTT:28000  TCPMetric:<nil> IPMetric: Start:1557936617439 Last:1557936617439 TrackingID:b9b515a863759060 L3TrackingID:34a355c715831971 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.394Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:56d63fb4cd956f69 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"de:56:b2:0a:9a:a8" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::dc56:b2ff:fe0a:9aa8" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936613439 Last:1557936618393  Metric:ABPackets:1 ABBytes:70 Start:1557936613439 Last:1557936613439  TCPMetric:<nil> IPMetric:<nil> Start:1557936613439 Last:1557936613439 TrackingID:be3fd8da376a363d L3TrackingID:27a834cddd0e848f ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:18.394Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:cd624bf33f455936 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"a2:5f:54:48:f3:b1" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::a05f:54ff:fe48:f3b1" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936613952 Last:1557936618393  Metric:ABPackets:1 ABBytes:70 Start:1557936613952 Last:1557936613952  TCPMetric:<nil> IPMetric:<nil> Start:1557936613952 Last:1557936613952 TrackingID:00aaf279c65ca7a5 L3TrackingID:c411d6b13e33f207 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:20.435Z	DEBUG	analyzer/flow_server.go:210 (*FlowServer).storeFlows	analyzer-1: 15 flows stored
2019-05-15T16:10:21.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:bfb83fc881b3a7c6 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"ce:a9:cc:5a:72:fe" B:"e2:61:85:03:4b:2c"  Network:Protocol:IPV4 A:"10.0.3.1" B:"10.0.1.1"  Transport:<nil> ICMP:Type:ECHO ID:21152  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936617439 Last:1557936621325 RTT:124000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936617439 Last:1557936617439 RTT:124000  TCPMetric:<nil> IPMetric: Start:1557936617439 Last:1557936617439 TrackingID:b9b515a863759060 L3TrackingID:34a355c715831971 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:21.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:664cc440d2a12242 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"ce:a9:cc:5a:72:fe" B:"e2:61:85:03:4b:2c"  Network:Protocol:IPV4 A:"10.0.3.1" B:"10.0.1.1"  Transport:<nil> ICMP:Type:ECHO ID:21171  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936618557 Last:1557936621325 RTT:156000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936618557 Last:1557936618558 RTT:156000  TCPMetric:<nil> IPMetric: Start:1557936618557 Last:1557936618558 TrackingID:4dbe9ebfaa7da553 L3TrackingID:345e71f416eb74a1 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:21.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:fbcb3453920415a0 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"ce:a9:cc:5a:72:fe" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::cca9:ccff:fe5a:72fe" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936616325 Last:1557936621325  Metric:ABPackets:3 ABBytes:210 Start:1557936606719 Last:1557936618047  TCPMetric:<nil> IPMetric:<nil> Start:1557936606719 Last:1557936618047 TrackingID:a1ee67a15987c4c0 L3TrackingID:66975eade8d9e1d2 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:21.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:2ead1bc380c33393 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"ae:fe:9b:c1:90:e2" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936616325 Last:1557936621325  Metric:ABPackets:9 ABBytes:477 Start:1557936606958 Last:1557936619283  TCPMetric:<nil> IPMetric:<nil> Start:1557936606958 Last:1557936619283 TrackingID:47b108140bcd1f6d L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:21.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:af4e60ee2ee60245 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"ae:fe:9b:c1:90:e2" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::acfe:9bff:fec1:90e2" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936616325 Last:1557936621325  Metric:ABPackets:2 ABBytes:140 Start:1557936610367 Last:1557936619071  TCPMetric:<nil> IPMetric:<nil> Start:1557936610367 Last:1557936619071 TrackingID:134e0993dc145d21 L3TrackingID:4d1954f153cda53e ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:21.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:84ccdc30c6266898 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"5a:4a:c0:33:49:98" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::584a:c0ff:fe33:4998" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936616325 Last:1557936621325  Metric:ABPackets:2 ABBytes:140 Start:1557936611071 Last:1557936618559  TCPMetric:<nil> IPMetric:<nil> Start:1557936611071 Last:1557936618559 TrackingID:3128f626dec4d6f8 L3TrackingID:e173bd00964a8d95 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:21.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:ff0e3bd422e52bc7 LayersPath:Ethernet/ARP Application:ARP Link:A:"ce:a9:cc:5a:72:fe" B:"ff:ff:ff:ff:ff:ff"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936616311 Last:1557936621325  Metric:ABPackets:1 ABBytes:42 Start:1557936616311 Last:1557936616311  TCPMetric:<nil> IPMetric:<nil> Start:1557936616311 Last:1557936616311 TrackingID:65001b5d84e20089 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:21.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:dcd3cd43bf3c6b94 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"ce:a9:cc:5a:72:fe" B:"e2:61:85:03:4b:2c"  Network:Protocol:IPV4 A:"10.0.3.1" B:"10.0.1.1"  Transport:<nil> ICMP:Type:ECHO ID:21193  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936619683 Last:1557936621325 RTT:131000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936619683 Last:1557936619683 RTT:131000  TCPMetric:<nil> IPMetric: Start:1557936619683 Last:1557936619683 TrackingID:ebbc87ba5cf38021 L3TrackingID:6ac8c091a16da26b ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:21.326Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:0013d28fda938e61 LayersPath:Ethernet/ARP Application:ARP Link:A:"e2:61:85:03:4b:2c" B:"ce:a9:cc:5a:72:fe"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42 Start:1557936616313 Last:1557936621325  Metric:ABPackets:1 ABBytes:42 Start:1557936616313 Last:1557936616313  TCPMetric:<nil> IPMetric:<nil> Start:1557936616313 Last:1557936616313 TrackingID:b15487ce81f032e6 L3TrackingID:5752b0859882550f ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:21.327Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:17ca1032de7940b2 LayersPath:Ethernet/IPv4/ICMPv4 Application:ICMPv4 Link:A:"ce:a9:cc:5a:72:fe" B:"e2:61:85:03:4b:2c"  Network:Protocol:IPV4 A:"10.0.3.1" B:"10.0.1.1"  Transport:<nil> ICMP:Type:ECHO ID:21135  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936616313 Last:1557936621325 RTT:997000  Metric:ABPackets:1 ABBytes:98 BAPackets:1 BABytes:98 Start:1557936616313 Last:1557936616314 RTT:997000  TCPMetric:<nil> IPMetric: Start:1557936616313 Last:1557936616314 TrackingID:e61b81694ed446ab L3TrackingID:c746ce95f8c08654 ParentUUID: NodeTID:50b44850-123b-590e-7353-ecebd9b4c225 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:23.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e377a17e74e58ab2 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"a2:5f:54:48:f3:b1" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::a05f:54ff:fe48:f3b1" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936618391 Last:1557936623391  Metric:ABPackets:2 ABBytes:140 Start:1557936613952 Last:1557936622655  TCPMetric:<nil> IPMetric:<nil> Start:1557936613952 Last:1557936622655 TrackingID:00aaf279c65ca7a5 L3TrackingID:c411d6b13e33f207 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:23.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:0510c35fb4fa2fd9 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"5a:4a:c0:33:49:98" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::584a:c0ff:fe33:4998" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936618391 Last:1557936623391  Metric:ABPackets:2 ABBytes:140 Start:1557936611071 Last:1557936618559  TCPMetric:<nil> IPMetric:<nil> Start:1557936611071 Last:1557936618559 TrackingID:3128f626dec4d6f8 L3TrackingID:e173bd00964a8d95 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:23.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:e19953540b36f848 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"4e:5d:de:31:7f:d9" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:3 ABBytes:159 Start:1557936618391 Last:1557936623391  Metric:ABPackets:32 ABBytes:1696 Start:1557936559227 Last:1557936622288  TCPMetric:<nil> IPMetric:<nil> Start:1557936559227 Last:1557936622288 TrackingID:95a343d976ba3ebe L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:23.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:ef3c410a8b7d013b LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"ce:a9:cc:5a:72:fe" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::cca9:ccff:fe5a:72fe" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936618391 Last:1557936623391  Metric:ABPackets:2 ABBytes:140 Start:1557936610432 Last:1557936618047  TCPMetric:<nil> IPMetric:<nil> Start:1557936610432 Last:1557936618047 TrackingID:a1ee67a15987c4c0 L3TrackingID:66975eade8d9e1d2 ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:23.392Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:26ba0e3b1f16da74 LayersPath:Ethernet/IPv6/ICMPv6/ICMPv6RouterSolicitation Application:ICMPv6RouterSolicitation Link:A:"de:56:b2:0a:9a:a8" B:"33:33:00:00:00:02"  Network:Protocol:IPV6 A:"fe80::dc56:b2ff:fe0a:9aa8" B:"ff02::2"  Transport:<nil> ICMP:Type:ROUTER  DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:70 Start:1557936618391 Last:1557936623391  Metric:ABPackets:2 ABBytes:140 Start:1557936613439 Last:1557936622655  TCPMetric:<nil> IPMetric:<nil> Start:1557936613439 Last:1557936622655 TrackingID:be3fd8da376a363d L3TrackingID:27a834cddd0e848f ParentUUID: NodeTID:7b64c037-543b-5029-42ee-6ac75cab0c49 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:23.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:eb00921c15bc9380 LayersPath:Ethernet/LLC/STP Application:STP Link:A:"66:3b:f6:27:8c:8f" B:"01:80:c2:00:00:00"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:2 ABBytes:106 Start:1557936618393 Last:1557936623392  Metric:ABPackets:31 ABBytes:1643 Start:1557936559707 Last:1557936621782  TCPMetric:<nil> IPMetric:<nil> Start:1557936559707 Last:1557936621782 TrackingID:7cfa23dc758b6d05 L3TrackingID:e5f4602a7f712767 ParentUUID: NodeTID:0a6a3a75-727e-574d-5ccd-25682b2386a4 LastRawPackets:[] RawPacketsCaptured:0 FinishType:NOT_FINISHED}
2019-05-15T16:10:23.393Z	DEBUG	analyzer/flow_server.go:108 (*FlowServerWebSocketConn).OnMessage	analyzer-1: New flow from Websocket connection: {XXX_state:{lastMetric:<nil> rtt1stPacket:0 updateVersion:0} UUID:393d9d80fb7c1991 LayersPath:Ethernet/ARP Application:ARP Link:A:"e2:61:85:03:4b:2c" B:"ce:a9:cc:5a:72:fe"  Network:<nil> Transport:<nil> ICMP:<nil> DHCPv4:<nil> DNS:<nil> VRRPv2:<nil> LastUpdateMetric:ABPackets:1 ABBytes:42if [ $status -ne 0 ] ; then
   echo "test Scale TLS:${TLS} FLOW_PROTOCOL:${FLOW_PROTOCOL} failed return ${status}"
   exit $status
fi

exit 0
Setting status of cea8588ce92f3a0867909087e38b944e42d8e02b to SUCCESS with url http://ci.skydive.network:8080/job/skydive-scale-tests/3248/ and message: 'Build finished. '
Using context: skydive-scale-tests
Setting status of cea8588ce92f3a0867909087e38b944e42d8e02b to SUCCESS with url http://ci-logs.skydive.network/skydive-scale-tests/builds/3248/log.html and message: ' '
Using context: skydive-scale-tests