Anygility for Tangled

This software is intended to create and use playbooks as described on the paper
"Anycast Agility: Network Playbooks to Fight DDoS".

Software installation instructions on Tangled testbed in INSTALL.TXT

This set of tools provided here is part of the Tangled anycast testbed
(https://anycast-testbed.nl). Even though most of the tools are tailored to our anycast network,
the tools could be easily customized to any anycast network. Check the script tangler-cli.py to
update the configuration as properly. 


This package have the following files/directories
src
├── 00-functions.sh
├── 00-globalvar.sh
├── 01-baseline.sh
├── 02-positive_prepend.sh
├── 03-negative_prepend.sh
├── 04-communities.sh
├── 05-poison.sh
├── INSTALL.TXT
├── README.TXT
├── dataset
├── pyreqs
│   ├── bgptuner-requirements.txt
│   ├── makeplaybook-requirements.txt
│   ├── runplaybook-requirements.txt
│   ├── tanglercli-requirements.txt
│   └── vpcli-requirements.txt
├── toolbox
│   ├── GeoLite2-ASN.mmdb
│   ├── GeoLite2-Country.mmdb
│   ├── attack-load.txt
│   ├── hitlist_example.txt
│   ├── ip2location.bin
│   └── verfploeter
└── tools
    ├── bgp-tuner
    │   ├── assets
    │   │   ├── custom.css
    │   │   └── sand-logo.png
    │   ├── bgp-tuner.py
    │   ├── dataset
    │   │   ├── fake.csv
    │   │   ├── prepend+withdraw-dataframe.csv
    │   │   ├── prepend-only-dataframe.csv
    │   │   └── prepend-only-dataframe_percent.csv
    │   ├── readme.md
    │   └── requirements.txt
    ├── make-playbook.py
    ├── run-playbook.py
    ├── tangler-cli.py
    └── vp-cli.py

=================== DEPENDENCIES ===================
1) Measurement scripts - starting by numbers on SRC directory. Need an anycast
   network access (here we used Tangled). This anycast network expects exabgp
   BGP sections established).
   Verfploter software also needs to be installed. An IP Hitlist need to be  provided.
   Each software path needs to be configured on 00-globalvar.sh.
2) Tangled-cli: Controls routing on Tangled testbed (https://anycast-testbed.nl).
3) vp-cli.py: need verfploeter raw files to build several metadata (ex.stats)
4) make-playbook.py: need metatada produced by vp-cli to build anycast playbook
5) run-playbook.py: need access to anycast testbed, anycast playbook, and routing files
6) bgp-tuner has its own requirements (install requirement from pyreqs directory).
   The graphical interface is customized for a set of anycast sites. 
   It is ready to use for the playbooks inside bgp-tuner/dataset folder.
7) Other dependencies software as geolocation files and hitlist need to be downloaded.
   We provided some examples on toolboox directory.

=================== PYTHON REQUIREMENT  ===================
You cant find all python requirement files in pyreqs directory. You need it to 
run this software. Follow an example of use:

pip install -r anygility-tangled/pyreqs/bgptuner-requirements.txt

=================== EXAMPLE OF SOFTWARE USE ===================
How to run (considering access to tangled testbed):
configure your paths at ./00-globalvar.sh
run ./01-baseline.sh
run ./make-playbook --dir dataset/last --out dataset/playbook.csv --fsdb dataset/playbook.fsdb

The baseline dataset provides a baseline measurement using just 3 nodes (br-poa, us-mia, and nl-ams).
the measurement process just used 0.01% of the full histlist as a way to try this tools sets and get
results faster. After script execution with this short-hitlist we got the following files:

➜  dataset git:(master) ✗ ls -1 baseline
baseline#ipv4,us-mia,br-poa,nl-ams#2022-02-28-15h02m.csv.gz         ---> Verfploter raw file
baseline#ipv4,us-mia,br-poa,nl-ams#2022-02-28-15h02m.meta           ---> vp-cli parameters used do generate stats
baseline#ipv4,us-mia,br-poa,nl-ams#2022-02-28-15h02m.meta-convert   ---> vp-cli params to convert file (weight,geoloc)
baseline#ipv4,us-mia,br-poa,nl-ams#2022-02-28-15h02m.routing        ---> routing information generated by Tangler-cli
baseline#ipv4,us-mia,br-poa,nl-ams#2022-02-28-15h02m.stats          ---> stats information generated by vp-cli
log.txt                                                             ---> script logs 


=== RUNNING VP-CLI.PY ===

vp-cli is called inside scripts to generate statistics from verfploeter. Command line arguments are registered in 
meta file. meta-convert file register command line used to generate normalized file (considering network weight and
more precise geolocation if user desires). 
In the normal vp-cli execution we can see a bar graph representation of catchment for rapid inspection.

➜  dataset git:(master) ✗ vp-cli.py -s us-mia-anycast01 -b \"baseline\" -f baseline/baseline\#ipv4,us-mia,br-poa,nl-ams\#2022-02-28-15h02m.csv.gz
nl-ams-anycast01 | 8449 -  94%  █████████████████████████
us-mia-anycast01 |  414 -   4%  █
br-poa-anycast02 |   51 -   0%  ▏

Inside the script we used parameter --csv to generate a csv file to be used to build the Playbook
➜  dataset git:(master) ✗ vp-cli.py -s us-mia-anycast01 -b \"baseline\" -f baseline/baseline\#ipv4,us-mia,br-poa,nl-ams\#2022-02-28-15h02m.csv.gz --csv
#policy,baseline
#timestamp,1646064121
#hitlist,not_provided
site,counts,percent
nl-ams-anycast01,8449,94
us-mia-anycast01,414,4
br-poa-anycast02,51,0

Scripts also invoke tangler-cli to register the current routing configuration used to do the measurement. This information 
is used to create a routing information file (.routing). Routing file stores each bgp peer configuration needed to set a 
specific BGP configuration on anycast network. Follow an example:

=== RUNNING TANGLER-CLI ====

➜  src git:(master) ✗ tangler-cli.py -a --csv
site,prefix,peer_as,neighbor,attributes
br-poa-anycast02,145.100.118.0/23,264575,177.52.38.113,
br-poa-anycast02,145.100.118.0/23,262605,177.184.254.161,
br-poa-anycast02,145.100.118.0/23,64552,145.100.119.1,
nl-ams-anycast01,145.100.118.0/23,64515,169.254.169.254,
us-mia-anycast01,145.100.118.0/23,20080,198.32.252.96,

Each running scripts also save This information on *.routing file.

➜  dataset git:(master) ✗ cat baseline/baseline\#ipv4,us-mia,br-poa,nl-ams\#2022-02-28-15h02m.routing
#policy,baseline
#ipv4,br-poa,us-mia,nl-ams
site,prefix,peer_as,neighbor,attributes
br-poa-anycast02,145.100.118.0/23,264575,177.52.38.113,
br-poa-anycast02,145.100.118.0/23,262605,177.184.254.161,
br-poa-anycast02,145.100.118.0/23,64552,145.100.119.1,
nl-ams-anycast01,145.100.118.0/23,64515,169.254.169.254,
us-mia-anycast01,145.100.118.0/23,20080,198.32.252.96,


=== Creating a playbook with MAKE-PLAYBOOK === 

➜  src git:(master) ✗ tools/make-playbook.py --dir dataset/baseline --dir dataset/bgp-community --dir dataset/negative-prepend --dir dataset/poison --dir dataset/prepend
Building Playbook from dataset/baseline
Building Playbook from dataset/bgp-community
Building Playbook from dataset/negative-prepend
Building Playbook from dataset/poison
Building Playbook from dataset/prepend
                             AMS   MIA  POA   CDG   LHR   SYD
bgp
baseline                    94.0   4.0  1.0   0.0   0.0   0.0
positive-20080:700xus-mia    0.0   0.0  0.0   0.0  10.0  89.0
positive-20080:701xus-mia    0.0   0.0  0.0   0.0  11.0  88.0
positive-20080:702xus-mia    0.0   0.0  0.0   0.0  10.0  89.0
positive-20473:6000xau-syd   0.0  24.0  0.0  50.0  24.0   0.0
...                          ...   ...  ...   ...   ...   ...
2xMIA                       98.0   0.0  0.0   0.0   0.0   0.0
2xPOA                       97.0   1.0  0.0   0.0   0.0   0.0
3xAMS                       72.0  18.0  9.0   0.0   0.0   0.0
3xMIA                       98.0   0.0  0.0   0.0   0.0   0.0
3xPOA                       98.0   1.0  0.0   0.0   0.0   0.0

NOTE: If you want to generate a playbook to run playbook_tuner you need to generate a playbook in FSDB format and need to include at leastthe baseline policy.
Example:

$ make-playbook.py --dir dataset/baseline/ --fsdb /tmp/f.fsdb
Building Playbook from dataset/baseline/
=== Playbook ===
site       bgp  CDG  LHR  SYD
0     baseline    5    5   89
================
Playbook FSDB saved to [/tmp/f.fsdb]

$ cat /tmp/f.fsdb
3
CDG	LHR	SYD
50000.0	50000.0	50000.0
Baseline	5	5	89


=== Setting up a playbook routing policy with RUN-PLAYBOOK ===

run-playbook.py read one routing file and setup BGP configuration accordingly.
➜  dataset git:(master) ✗run-playbook.py --playbook baseline/baseline\#ipv4,us-mia,br-poa,nl-ams\#2022-02-28-15h02m.routing

The make-playbook.py tool uses all information provided before to build the playbook. Playbook can be generated containing
just statistical information and BGP policy, or include a direct reference to routing file too (--routing option).
To make a playbook it is necessary just indicate the folder where script exacution stored files. In this case we provided some 
example measurements in dataset directory: baseline, bgp-community, negative-prepend, poison, and prepend. So to generate a 
playbook considering all that measurements, you can run:

➜  src git:(master) ✗ tools/run-playbook.py  --playbook dataset/baseline/baseline\#ipv4,us-mia,br-poa,nl-ams\#2022-02-28-16h10m.routing
== br-poa-anycast02
neighbor 177.52.38.113 prefix 145.100.118.0/23
neighbor 177.184.254.161 prefix 145.100.118.0/23
neighbor 145.100.119.1 prefix 145.100.118.0/23
== nl-ams-anycast01
neighbor 169.254.169.254 prefix 145.100.118.0/23
== us-mia-anycast01
neighbor 198.32.252.96 prefix 145.100.118.0/23
 Found [3] nodes

Any doubt about this software contact Leandro Bertholdo <leandro.bertholdo@gmail.com>>
