{"id":945,"date":"2019-03-05T21:32:36","date_gmt":"2019-03-05T21:32:36","guid":{"rendered":"https:\/\/psyphi.net\/blog\/?p=945"},"modified":"2019-03-06T09:07:55","modified_gmt":"2019-03-06T09:07:55","slug":"proxy-testing-with-ip-namespaces-and-gitlab-ci-cd","status":"publish","type":"post","link":"https:\/\/psyphi.net\/blog\/2019\/03\/proxy-testing-with-ip-namespaces-and-gitlab-ci-cd\/","title":{"rendered":"Proxy testing with IP Namespaces and GitLab CI\/CD"},"content":{"rendered":"\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"688\" src=\"https:\/\/psyphi.net\/wp-uploads\/2019\/03\/106559730_30946688a9_b-1024x688.jpg\" alt=\"\" class=\"wp-image-947\" srcset=\"https:\/\/psyphi.net\/wp-uploads\/2019\/03\/106559730_30946688a9_b.jpg 1024w, https:\/\/psyphi.net\/wp-uploads\/2019\/03\/106559730_30946688a9_b-300x202.jpg 300w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption>CC-BY-NC https:\/\/www.flickr.com\/photos\/thomashawk\/106559730<\/figcaption><\/figure>\n\n\n\n<p>At work, I have a CLI tool I&#8217;ve been working on. It talks to the web and is used by customers all over the planet, some of them on networks with tighter restrictions than my own. Often those customers have an HTTP proxy of some sort and that means the CLI application needs to negotiate with it differently than it would directly with a web server.<\/p>\n\n\n\n<p>So I need to test it somehow with a proxy environment. Installing a proxy service like <a href=\"http:\/\/www.squid-cache.org\/\">Squid<\/a> doesn&#8217;t sound like too big a deal but it needs to run in several configurations, at a very minimum these three:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>no-proxy<\/li><li>authenticating HTTP proxy<\/li><li>non-authenticating HTTP proxy<\/li><\/ul>\n\n\n\n<p>I&#8217;m going to ignore HTTPS proxy for now as it&#8217;s not actually a common configuration for customers but I reckon it&#8217;s possible to do with <g class=\"gr_ gr_4 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling\" id=\"4\" data-gr-id=\"4\"><a href=\"https:\/\/github.com\/FiloSottile\/mkcert\">mkcert<\/a><\/g> or <a href=\"https:\/\/letsencrypt.org\/\">LetsEncrypt<\/a> without too much work.<\/p>\n\n\n\n<p>There are two other useful pieces of information to cover, firstly I use <a href=\"https:\/\/about.gitlab.com\/\">GitLab-CI<\/a> to run the CI\/CD test stages for the three proxy configurations in parallel. Secondly, and this is important, I must make sure that, once the test Squid proxy service is running, the web requests in the test only pass through the proxy and do not leak out of the GitLab runner. I can do this by using a really neat Linux feature called <a href=\"https:\/\/blog.scottlowe.org\/2013\/09\/04\/introducing-linux-network-namespaces\/\">IP namespaces<\/a>.<\/p>\n\n\n\n<p>IP namespaces allow me to set up different network environments on the same machine, similar to <a href=\"https:\/\/en.wikipedia.org\/wiki\/Subnetwork\">IP subnets<\/a> or <a href=\"https:\/\/docs.aws.amazon.com\/vpc\/latest\/userguide\/VPC_SecurityGroups.html#VPCSecurityGroups\">AWS security groups<\/a>. Then I can launch specific processes in those namespaces and network access from those processes will be limited by the configuration of the network namespace. That is to say, the Squid proxy can have full access but the test process can <strong>only<\/strong> talk to the proxy. Cool, right?<\/p>\n\n\n\n<p>The GitLab CI\/CD YAML looks like this (edited to protect the innocent)<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">stages:<br \/> - integration<br \/> <br \/>.integration_common: &amp;integration_common |<br \/>   apt-get update<br \/>   apt-get install -y iproute2<br \/><br \/>.network_ns: &amp;network_ns |<br \/>   ip netns add $namespace<br \/>   ip link add v-eth1 type veth peer name v-peer1<br \/>   ip link set v-peer1 netns $namespace<br \/>   ip addr add 192.168.254.1\/30 dev v-eth1<br \/>   ip link set v-eth1 up<br \/>   ip netns exec $namespace ip addr add 192.168.254.2\/30 dev v-peer1<br \/>   ip netns exec $namespace ip link set v-peer1 up<br \/>   ip netns exec $namespace ip link set lo up<br \/>   ip netns exec $namespace ip route add default via 192.168.254.1<br \/><br \/>noproxynoauth-cli:<br \/>   image: ubuntu:18.04<br \/>   stage: integration<br \/>   script:<br \/>     - *integration_common<br \/>     - test\/end2end\/cli<br \/><br \/>proxyauth-cli:<br \/>   image: ubuntu:18.04<br \/>   stage: integration<br \/>   script:<br \/>     - *integration_common<br \/>     - apt-get install -y squid apache2-utils<br \/>     - mkdir -p \/etc\/squid3<br \/>     - htpasswd -cb \/etc\/squid3\/passwords testuser testpass<br \/>     - *network_ns<br \/>     - squid3 -f test\/end2end\/conf\/squid.conf.auth &amp;&amp; sleep 1 || tail -20 \/var\/log\/syslog | grep squid<br \/>     - http_proxy=http:\/\/testuser:testpass@192.168.254.1:3128\/ https_proxy=http:\/\/testuser:testpass@192.168.254.1:3128\/ ip netns exec $namespace test\/end2end\/cli<br \/>     - ip netns del $namespace || true<br \/>   variables:<br \/>     namespace: proxyauth<br \/><br \/>proxynoauth-cli:<br \/>   image: ubuntu:18.04<br \/>   stage: integration<br \/>   script:<br \/>     - *integration_common<br \/>     - apt-get install -y squid<br \/>     - *network_ns<br \/>     - squid3 -f test\/end2end\/conf\/squid.conf.noauth &amp;&amp; sleep 1 || tail -20 \/var\/log\/syslog | grep squid<br \/>     - http_proxy=http:\/\/192.168.254.1:3128\/ https_proxy=http:\/\/192.168.254.1:3128\/ test\/end2end\/cli<br \/>     - ip netns del $namespace || true<br \/>   variables:<br \/>     namespace: proxynoauth <\/pre>\n\n\n\n<p>So there are five blocks here, with three stages and two common script blocks. The first common script block installs iproute2 which gives us the <g class=\"gr_ gr_7 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace\" id=\"7\" data-gr-id=\"7\"><em>ip<\/em><\/g> command.<\/p>\n\n\n\n<p>The second script block is where the magic happens. It configures a virtual, routed subnet in the parameterised <em>$namespace<\/em>.<\/p>\n\n\n\n<p>Following that we have the three test stages corresponding to the three proxy (or not) configurations I listed earlier. Two of them install Squid, one of those creates a test user for authenticating with the proxy. They all run the test script, which in this case is <em>test\/end2end\/<\/em><g class=\"gr_ gr_6 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling\" id=\"6\" data-gr-id=\"6\"><em>cli<\/em><\/g>. When those three configs are modularised and out like this with the common net namespace script as well it provides a good deal of clarity to the test maintainer. I like it a lot.<\/p>\n\n\n\n<p>So then the last remaining things are the respective squid configurations: <g class=\"gr_ gr_4 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace\" id=\"4\" data-gr-id=\"4\"><em>proxyauth<\/em><\/g> and <g class=\"gr_ gr_5 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace\" id=\"5\" data-gr-id=\"5\"><em class=\"\">proxynoauth<\/em><\/g>. There&#8217;s a little bit more junk in these than there needs to be as they&#8217;re taken from the stock examples, but they look something like this:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"> visible_hostname proxynoauth<br \/> acl localnet src 10.0.0.0\/8 # RFC1918 possible internal network<br \/> acl localnet src 172.16.0.0\/12 # RFC1918 possible internal network<br \/> acl localnet src 192.168.0.0\/16 # RFC1918 possible internal network<br \/> acl SSL_ports port 443<br \/> acl Safe_ports port 80 # http<br \/> acl Safe_ports port 443 # https<br \/> acl CONNECT method CONNECT<br \/> http_access deny !Safe_ports<br \/> http_access deny CONNECT !SSL_ports<br \/> http_access allow localhost manager<br \/> http_access deny manager<br \/> http_access allow localnet<br \/> http_access allow localhost<br \/> http_access deny all<br \/> http_port 3128 <\/pre>\n\n\n\n<p>and for authentication:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"> visible_hostname proxyauth<br \/> acl localnet src 10.0.0.0\/8 # RFC1918 possible internal network<br \/> acl localnet src 172.16.0.0\/12 # RFC1918 possible internal network<br \/> acl localnet src 192.168.0.0\/16 # RFC1918 possible internal network<br \/> acl SSL_ports port 443<br \/> acl Safe_ports port 80 # http<br \/> acl Safe_ports port 443 # https<br \/> acl CONNECT method CONNECT<br \/> http_access deny !Safe_ports<br \/> http_access deny CONNECT !SSL_ports<br \/> http_access allow localhost manager<br \/> http_access deny manager<br \/> <br \/> auth_param basic program \/usr\/lib\/squid3\/basic_ncsa_auth \/etc\/squid3\/passwords<br \/> auth_param basic realm proxy<br \/> acl authenticated proxy_auth REQUIRED<br \/><br \/> http_access allow authenticated<br \/> http_access deny all<br \/> http_port 3128 <\/pre>\n\n\n\n<p>And there you have it &#8211; network-restricted proxy testing with different proxy configurations. It&#8217;s the first time I&#8217;ve used <em><g class=\"gr_ gr_3 gr-alert gr_spell gr_inline_cards gr_disable_anim_appear ContextualSpelling ins-del multiReplace\" id=\"3\" data-gr-id=\"3\">ip<\/g> net ns<\/em> without being wrapped up in <a href=\"https:\/\/www.docker.com\/\">Docker<\/a>, <a href=\"https:\/\/linuxcontainers.org\/\">LXC<\/a>, <g class=\"gr_ gr_4 gr-alert gr_spell gr_inline_cards gr_disable_anim_appear ContextualSpelling ins-del multiReplace\" id=\"4\" data-gr-id=\"4\"><a href=\"https:\/\/containerd.io\/\">containerd<\/a><\/g> or some other <g class=\"gr_ gr_5 gr-alert gr_spell gr_inline_cards gr_disable_anim_appear ContextualSpelling\" id=\"5\" data-gr-id=\"5\"><a href=\"https:\/\/libvirt.org\/\">libvirt<\/a><\/g> thing, but the feeling of power from my new-found network-god skills is quite something :)<\/p>\n\n\n\n<p>Be aware that you might need to choose different subnet ranges if your regular LAN conflicts. Please let me know in the comments if you find this useful or if you had to modify things to work in your environment.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>At work, I have a CLI tool I&#8217;ve been working on. It talks to the web and is used by customers all over the planet, some of them on networks with tighter restrictions than my own. Often those customers have an HTTP proxy of some sort and that means the CLI application needs to negotiate &hellip; <a href=\"https:\/\/psyphi.net\/blog\/2019\/03\/proxy-testing-with-ip-namespaces-and-gitlab-ci-cd\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Proxy testing with IP Namespaces and GitLab CI\/CD&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[366,17,1105,18,9],"tags":[1109,1108,1107,1106,199,1111,275,8,1110,219],"class_list":["post-945","post","type-post","status-publish","format-standard","hentry","category-security","category-sysadmin","category-testing","category-virtualisation","category-webdev","tag-ci-cd","tag-gitlab","tag-ip-net-ns","tag-libvirt","tag-linux","tag-namespaces","tag-networking","tag-proxy","tag-squid","tag-testing"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/posts\/945","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/comments?post=945"}],"version-history":[{"count":5,"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/posts\/945\/revisions"}],"predecessor-version":[{"id":951,"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/posts\/945\/revisions\/951"}],"wp:attachment":[{"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/media?parent=945"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/categories?post=945"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/psyphi.net\/blog\/wp-json\/wp\/v2\/tags?post=945"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}