Zola 2025-10-15T00:00:00+00:00 https://jasonish.org/atom.xml Quick Guide to Building and Testing Suricata on Linux 2025-10-15T00:00:00+00:00 2025-10-15T00:00:00+00:00 Unknown https://jasonish.org/blog/quick-guide-building-testing-suricata-linux/ <h3 id="target-audience">Target Audience</h3> <p>This quick guide is intended for those who would like to develop and/or make changes to Suricata. These instructions are not suitable for a live or production installation of Suricata.</p> <h3 id="requirements">Requirements</h3> <ul> <li>Fedora 42 or Ubuntu 24.04: Inside WSL is OK. Other versions of Fedora and Ubuntu might need the package installation commands modified.</li> <li>Root access via <code>sudo</code>.</li> <li>Basic familiarity with the Linux command line, <code>git</code> and GitHub.</li> </ul> <h3 id="install-dependencies">Install Dependencies</h3> <p>Before building Suricata, let's get a basic set of system dependencies installed for building and developing Suricata.</p> <div class="alert alert-info" role="alert"> ℹ️ This is the only part of the guide that is specific to your Linux distribution. Use the package installation commands for your distribution of Linux. </div><h4 id="for-fedora-42">For Fedora 42</h4> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">sudo</span><span> dnf</span><span style="color:#bf616a;"> -y</span><span> install autoconf automake bindgen cargo-vendor cbindgen diffutils dpdk-devel file-devel gcc git hwloc-devel jansson-devel jq libasan libcap-ng-devel libnet-devel libnetfilter_queue-devel libnfnetlink-devel libpcap-devel libtool libyaml-devel lz4-devel make numactl-devel pcre2-devel pkgconfig python3-devel python3-sphinx python3-yaml rust cargo which zlib-devel </span></code></pre> <h4 id="for-ubuntu-24-04">For Ubuntu 24.04</h4> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">sudo</span><span> apt update &amp;&amp; </span><span style="color:#bf616a;">sudo</span><span> DEBIAN_FRONTEND=noninteractive apt</span><span style="color:#bf616a;"> -y</span><span> install autoconf automake bindgen build-essential cargo cbindgen curl git jq libcap-ng-dev libhyperscan-dev libjansson-dev libmagic-dev libnet1-dev libnetfilter-queue-dev libnetfilter-queue1 libnfnetlink-dev libnfnetlink0 libnuma-dev libpcap-dev libpcre2-dev libtool libyaml-dev make python-is-python3 python3-yaml rustc software-properties-common zlib1g zlib1g-dev </span></code></pre> <h3 id="checkout-suricata-and-suricata-verify">Checkout Suricata and Suricata-Verify</h3> <p>Now it's time to check out Suricata:</p> <ul> <li>To keep this guide brief, we are cloning from the OISF GitHub repositories. If you plan to contribute back to Suricata, you should first create your own forks of these repositories on GitHub then check out your forks.</li> <li>This guide assumes the use of the directory <code>~/suridev</code> for your Suricata development. Change as needed.</li> </ul> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">mkdir -p ~</span><span>/suridev </span><span style="color:#96b5b4;">cd </span><span style="color:#bf616a;">~</span><span>/suridev </span><span style="color:#bf616a;">git</span><span> clone https://github.com/OISF/suricata </span><span style="color:#bf616a;">git</span><span> clone https://github.com/OISF/suricata-verify </span></code></pre> <h3 id="build-suricata-for-development">Build Suricata for Development</h3> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#96b5b4;">cd </span><span style="color:#bf616a;">~</span><span>/suridev/suricata </span><span style="color:#bf616a;">./autogen.sh </span><span style="color:#bf616a;">CFLAGS</span><span>=&quot;</span><span style="color:#a3be8c;">-fsanitize=address</span><span>&quot; </span><span style="color:#bf616a;">./configure --enable-unittests </span><span style="color:#bf616a;">make -j4 </span></code></pre> <p>Notes:</p> <ul> <li>The <code>-j4</code> option to <code>make</code> specifies the parallelization level. If you have 16 cores to spare, feel free to run <code>make -j16</code>, and so on.</li> </ul> <h3 id="run-unit-tests">Run Unit Tests</h3> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">./src/suricata -l</span><span> .</span><span style="color:#bf616a;"> -u </span></code></pre> <p>Notes:</p> <ul> <li>To run a subset of tests, use the <code>-U</code> command-line option with a regular expression or substring, for example: <code>-U DNS</code>.</li> <li>To quit immediately after a unit test fails, provide <code>--fatal-unittests</code> on the command line.</li> </ul> <h3 id="run-verification-tests">Run Verification Tests</h3> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">../suricata-verify/run.py -q </span></code></pre> <p>Notes:</p> <ul> <li>It can take many minutes to run all the verification tests.</li> <li>To run only tests whose name matches your filter, add a test name (or partial name) to the command line, for example: <code>../suricata-verify/run.py -q dns</code>.</li> <li>Leave off the <code>-q</code> to see a line item for each test.</li> <li>To quit immediately after a test fails, add <code>--fail</code>.</li> </ul> <h3 id="demo">Demo</h3> <p>Demo of the above commands on Ubuntu 24.04.</p> <script src="https://asciinema.org/a/Lm2N0faDskirySyVOM00pEmyB.js" id="asciicast-Lm2N0faDskirySyVOM00pEmyB" async="true"></script> <p>Alternative versions: <a href="/images/fast-guide-building-testing-suricata-ubuntu/suricata-build-testing-demo.mp4">mp4</a> <a href="/images/fast-guide-building-testing-suricata-ubuntu/suricata-build-testing-demo.gif">gif</a></p> <h3 id="next-steps">Next Steps</h3> <h4 id="break-a-suricata-verify-test">Break a Suricata-Verify Test</h4> <ul> <li>Modify a Suricata source file such that the output will change. For example, in <code>rust/src/ike/logger.rs</code>, remove <code>init_spi</code> from the output (hint: look for <code>jb.set_string("init_spi"</code>).</li> <li>Re-build Suricata: <code>make</code></li> <li>Run the IKE Suricata-Verify tests: <code>../suricata-verify/run.py ike</code></li> </ul> <p>Notice how the test fails because we changed the output from what the verification test expects.</p> <h4 id="create-your-own-suricata-verify-test">Create Your Own Suricata-Verify Test</h4> <p>A good way to start here is to make a copy of an existing test that is similar to what you are working on. Any test will do for the sake of example:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#96b5b4;">cd </span><span style="color:#bf616a;">~</span><span>/suridev/suricata-verify/tests </span><span style="color:#bf616a;">cp -a</span><span> alert-testmyids my-awesome-test </span></code></pre> <p>Now run your new test:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#96b5b4;">cd </span><span style="color:#bf616a;">~</span><span>/suridev/suricata </span><span style="color:#bf616a;">../suricata-verify/run.py</span><span> my-awesome-test </span></code></pre> Quick Guide to Building and Testing Suricata on Ubuntu 2025-10-14T00:00:00+00:00 2025-10-14T00:00:00+00:00 Unknown https://jasonish.org/blog/quick-guide-building-testing-suricata-ubuntu/ <div class="alert alert-info" role="alert"> <strong>Note:</strong> This guide has been superseded by <a href="/blog/quick-guide-building-testing-suricata-linux">Quick Guide to Building and Testing Suricata on Linux</a>, which covers both Ubuntu and Fedora distributions. </div><h3 id="target-audience">Target Audience</h3> <p>This quick guide is intended for those who would like to develop and/or make changes to Suricata. These instructions are not suitable for a live or production installation of Suricata.</p> <h3 id="requirements">Requirements</h3> <ul> <li>Ubuntu 24.04 (inside WSL is <strong>OK</strong>)</li> <li>Root access via <code>sudo</code>.</li> <li>Basic familiarity with the Linux command line, <code>git</code> and GitHub.</li> </ul> <h3 id="install-dependencies">Install Dependencies</h3> <p>Before building Suricata, let's get a basic set of system dependencies installed for building and developing Suricata.</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">sudo</span><span> apt update &amp;&amp; </span><span style="color:#bf616a;">sudo</span><span> DEBIAN_FRONTEND=noninteractive apt</span><span style="color:#bf616a;"> -y</span><span> install autoconf automake bindgen build-essential cargo cbindgen curl git jq libcap-ng-dev libhyperscan-dev libjansson-dev libmagic-dev libnet1-dev libnetfilter-queue-dev libnetfilter-queue1 libnfnetlink-dev libnfnetlink0 libnuma-dev libpcap-dev libpcre2-dev libtool libyaml-dev make python-is-python3 python3-yaml rustc software-properties-common zlib1g zlib1g-dev </span></code></pre> <h3 id="checkout-suricata-and-suricata-verify">Checkout Suricata and Suricata-Verify</h3> <p>Now it's time to check out Suricata:</p> <ul> <li>To keep this guide brief, we are cloning from the OISF GitHub repositories. If you plan to contribute back to Suricata, you should first create your own forks of these repositories on GitHub then check out your forks.</li> <li>This guide assumes the use of the directory <code>~/suridev</code> for your Suricata development. Change as needed.</li> </ul> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">mkdir -p ~</span><span>/suridev </span><span style="color:#96b5b4;">cd </span><span style="color:#bf616a;">~</span><span>/suridev </span><span style="color:#bf616a;">git</span><span> clone https://github.com/OISF/suricata </span><span style="color:#bf616a;">git</span><span> clone https://github.com/OISF/suricata-verify </span></code></pre> <h3 id="build-suricata-for-development">Build Suricata for Development</h3> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#96b5b4;">cd </span><span style="color:#bf616a;">~</span><span>/suridev/suricata </span><span style="color:#bf616a;">./autogen.sh </span><span style="color:#bf616a;">CFLAGS</span><span>=&quot;</span><span style="color:#a3be8c;">-fsanitize=address</span><span>&quot; </span><span style="color:#bf616a;">./configure --enable-unittests </span><span style="color:#bf616a;">make -j4 </span></code></pre> <p>Notes:</p> <ul> <li>The <code>-j4</code> option to <code>make</code> specifies the parallelization level. If you have 16 cores to spare, feel free to run <code>make -j16</code>, and so on.</li> </ul> <h3 id="run-unit-tests">Run Unit Tests</h3> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">ASAN_OPTIONS</span><span>=</span><span style="color:#a3be8c;">detect_leaks=0 </span><span style="color:#bf616a;">./src/suricata -l</span><span> .</span><span style="color:#bf616a;"> -u </span></code></pre> <p>Notes:</p> <ul> <li>To run a subset of tests, use the <code>-U</code> command-line option with a regular expression or substring, for example: <code>-U DNS</code>.</li> <li>To quit immediately after a unit test fails, provide <code>--fatal-unittests</code> on the command line.</li> <li>We hope to eliminate the need for <code>ASAN_OPTIONS=detect_leaks=0</code> soon.</li> </ul> <h3 id="run-verification-tests">Run Verification Tests</h3> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">../suricata-verify/run.py -q </span></code></pre> <p>Notes:</p> <ul> <li>It can take many minutes to run all the verification tests.</li> <li>To run only tests whose name matches your filter, add a test name (or partial name) to the command line, for example: <code>../suricata-verify/run.py -q dns</code>.</li> <li>Leave off the <code>-q</code> to see a line item for each test.</li> <li>To quit immediately after a test fails, add <code>--fail</code>.</li> </ul> <h3 id="demo">Demo</h3> <script src="https://asciinema.org/a/Lm2N0faDskirySyVOM00pEmyB.js" id="asciicast-Lm2N0faDskirySyVOM00pEmyB" async="true"></script> <p>Alternative versions: <a href="/images/fast-guide-building-testing-suricata-ubuntu/suricata-build-testing-demo.mp4">mp4</a> <a href="/images/fast-guide-building-testing-suricata-ubuntu/suricata-build-testing-demo.gif">gif</a></p> <h3 id="next-steps">Next Steps</h3> <h4 id="break-a-suricata-verify-test">Break a Suricata-Verify Test</h4> <ul> <li>Modify a Suricata source file such that the output will change. For example, in <code>rust/src/ike/logger.rs</code>, remove <code>init_spi</code> from the output (hint: look for <code>jb.set_string("init_spi"</code>).</li> <li>Re-build Suricata: <code>make</code></li> <li>Run the IKE Suricata-Verify tests: <code>../suricata-verify/run.py ike</code></li> </ul> <p>Notice how the test fails because we changed the output from what the verification test expects.</p> <h4 id="create-your-own-suricata-verify-test">Create Your Own Suricata-Verify Test</h4> <p>A good way to start here is to make a copy of an existing test that is similar to what you are working on. Any test will do for the sake of example:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#96b5b4;">cd </span><span style="color:#bf616a;">~</span><span>/suridev/suricata-verify/tests </span><span style="color:#bf616a;">cp -a</span><span> alert-testmyids my-awesome-test </span></code></pre> <p>Now run your new test:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#96b5b4;">cd </span><span style="color:#bf616a;">~</span><span>/suridev/suricata </span><span style="color:#bf616a;">../suricata-verify/run.py</span><span> my-awesome-test </span></code></pre> Suricata Test Builders 2017-11-23T18:43:32+00:00 2017-11-23T18:43:32+00:00 Unknown https://jasonish.org/blog/suricata-test-builders/ <div class="alert alert-warning" role="alert"> This post is out of date and references images and links that no longer exist. </div> <p>Do your Suricata modifications build for you but fail on the <a href="https://travis-ci.org/OISF/suricata">Suricata Travis-CI</a>? <img alt="travis-suricata-failed-jobs" src="/images/travis-suricata-failed-jobs.png" height="409" width="833" /></p> <p>Or does your PR get rejected for not building on the private build servers? Try <a href="https://github.com/jasonish/suricata-test-builders">suricata-test-builders</a>, an unofficial set of Docker and Vagrant build environments that perform a variety of builds across a variety of Linux distributions (with Docker) and other operating systems with Vagrant and VirtualBox. The idea is to run these builders from your current working directory to exercise a variety of build environment including CentOS, Fedora, Debian, Ubuntu, FreeBSD and OpenBSD with an option of providing you a shell for further debugging. As Docker and Vagrant/VirtualBox are used it is limited to x86 and x64 systems that can run under Docker and VirtualBox. Most testing has been done on Linux, but it should also work on MacOS provided VirtualBox, Vagrant and Docker are installed and working. <a href="https://github.com/jasonish/suricata-test-builders">https://github.com/jasonish/suricata-test-builders</a></p> EveBox Oneshot Demo 2017-07-18T20:15:26+00:00 2017-07-18T20:15:26+00:00 Unknown https://jasonish.org/blog/evebox-oneshot-demo/ <p>Following up on EveBox support for <a href="http://blog.jasonish.org/2017/04/03/evebox-with-sqlite/">SQLite</a>, I've created a screencast demo of <a href="http://evebox.readthedocs.io/en/latest/server-oneshot.html">oneshot </a>mode.</p> <p>Oneshot is an interactive run of EveBox to process a single eve.json file and view it in the EveBox UI.</p> <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;"> <iframe src="https://www.youtube.com/embed/nbWhci3ed80" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe> </div> EveBox with SQLite 2017-04-03T22:50:48+00:00 2017-04-03T22:50:48+00:00 Unknown https://jasonish.org/blog/evebox-with-sqlite/ <p>The latest builds of EveBox support an embedded SQLite database that allow it to be used without Elastic Search for lighter loads. The SQLite support was added to support two use cases that may be of interest to some.</p> <h3 id="one-shot-mode">One Shot Mode</h3> <p>One shot mode is the loading of a single eve.json into a temporary database and allowing the user to work with it, then cleaning up on exit. Probably most useful for loading up the Suricata log file after running over a PCAP, or just trying out EveBox for the first time. Example usage:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>./evebox oneshot /path/to/eve.json </span></code></pre> <p>If all goes well your browser should eventually open up and display the EveBox Inbox.</p> <h3 id="self-contained-mode">Self Contained Mode</h3> <p>For lack of a better name, self contained mode is the usage of EveBox without any external dependencies. This is suitable for lighter loads when running EveBox on the same machine that is running Suricata. Example usage:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>./evebox server --datastore sqlite --input /var/log/suricata/eve.log </span></code></pre> <p>The idea here is just a simple way to get a GUI for your Suricata events without messing around with any configuration or databases. However, you may want to create a <a href="https://github.com/jasonish/evebox/blob/0.6.1/evebox.yaml.example">configuration file</a> and setup a <a href="https://github.com/jasonish/evebox/blob/0.6.1/evebox.yaml.example#L34">retention period</a> to keep your SQLite database size in check (more documentation coming soon). If you have multiple Suricata instances, and believe the load to be light, you can configure an <a href="http://evebox.readthedocs.io/en/latest/agent.html">EveBox agent</a> to send events to the SQLite enabled server, but your mileage will vary as you add more load.</p> <h3 id="using-elastic-search">Using Elastic Search?</h3> <p>If using Elastic Search the <a href="http://evebox.readthedocs.io/en/latest/agent.html">agent</a> and/or the <em><strong>--input</strong></em> option may still be interesting as alternatives for shipping eve logs to Elastic Search, and open up future options for dealing with the real time event feeds from your Suricata instances. Download <a href="https://evebox.org/">here</a>. <img alt="inbox" src="/images/inbox.png" height="740" width="885" /></p> EveBox - Stable Repositories 2017-03-16T15:44:23+00:00 2017-03-16T15:44:23+00:00 Unknown https://jasonish.org/blog/evebox-stable-repositories/ <p>I've been asked a few times now for "stable" APT and Yum repositories as the current ones are marked "development", in fact they contain the packages created on <a href="https://travis-ci.org/jasonish/evebox/branches">Travis-CI</a> runs of the master branch. So I've added stable repos for Yum and Apt. For the short term they still contain builds out of the master branch, but uploaded by me instead of the output CI, and they will transition to only tagged releases after the next release, 0.6.0 which I will probably tag soon.</p> <ul> <li><a href="https://github.com/jasonish/evebox/wiki/EveBox-RPM-Repository">Yum/DNF RPM Repository Setup</a></li> <li><a href="https://github.com/jasonish/evebox/wiki/EveBox-Debian-Repository">Apt Repository Setup</a></li> </ul> <p>The builds in the repos above should work with any modern x86_64 Fedora, CentOS, EL, Debian or Ubuntu distribution. And if you'd rather just get at the files, I've made it a little easier than the Bintray URLs make it -- <a href="https://evebox.org/files/development/">https://evebox.org/files/development/</a>.</p> eve2pcap - Eve Packet and Payload Conversion to pcap 2015-10-01T22:42:53+00:00 2015-10-01T22:42:53+00:00 Unknown https://jasonish.org/blog/eve2pcap-eve-packet-and-payload-conversion-to-pcap/ <p>I've added a new tool to my <a href="https://github.com/jasonish/py-idstools">idstools</a> package to convert the packets (or the payloads) found in Suricata eve logs to a pcap file. To just grab the script, download <a href="https://raw.githubusercontent.com/jasonish/py-idstools/master/idstools/scripts/eve2pcap.py">eve2pcap.py</a> and make it executable, or to install the complete idstools package (will install as <em>idstools-eve2pcap</em>):</p> <blockquote> <p>pip install --upgrade idstools</p> </blockquote> <p>Usage is pretty simple:</p> <blockquote> <p>./eve2pcap.py -o output.pcap /path/to/eve.json</p> </blockquote> <p>Or to use the payload field instead of the packet field:</p> <blockquote> <p>./eve2pcap.py -o output.pcap --payload /path/to/eve.json</p> </blockquote> <p>For straight packet conversion no dependencies are required other than Python and libpcap. Scapy is used for conversion of the payload field, so make sure to install it before trying to convert the payload. It is also important to note that eve logs do not contain all the information to recreate the packet headers, so when converting payloads to pcap the headers are "manufactured" and may not always produce the best packet for the payload, so YMMV.</p> Suricata Quick Start for Fedora 21 and 22 2015-06-01T22:47:36+00:00 2015-06-01T22:47:36+00:00 Unknown https://jasonish.org/blog/suricata-quick-start-for-fedora-21-and-22/ <p>Fedora Linux 21 and 22 have Suricata 2.0.8 in their default repositories making it very easy to get started. The following was done on Fedora 22, but should be applicable to Fedora 21 as well.</p> <h3 id="install-suricata">Install Suricata</h3> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install suricata </span></code></pre> <h3 id="configure-interface">Configure Interface</h3> <p>By default, Suricata will be configured to run on eth0, if you need to change this, edit <em>/etc/sysconfig/suricata</em> and change <em>eth0</em> to your desired interface.</p> <h3 id="start-suricata">Start Suricata</h3> <p>To start Suricata one time:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>systemctl start suricata </span></code></pre> <p>To have it restarted on each boot:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>systemctl enable suricata </span></code></pre> <p>But you'll probably want to have it started <strong>after</strong> your network is ready. This can be done by including <em>network-online.target</em> in the unit file <em>/usr/lib/systemd/system/suricata.service</em></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[Unit] </span><span>Description=Suricata Intrusion Detection Service </span><span>After=syslog.target network-online.target </span><span> </span><span>[Service] </span><span>ExecStart=/sbin/suricata -c /etc/suricata/suricata.yaml $OPTIONS </span><span>EnvironmentFile=-/etc/sysconfig/suricata </span><span> </span><span>[Install] </span><span>WantedBy=multi-user.target </span><span> </span></code></pre> <h3 id="verify-that-suricata-is-running">Verify That Suricata is Running</h3> <p>Even though we have not downloaded any rules yet, Suricata will still log HTTP requests, DNS requests, TLS certificates and SSH connection by default. These can be observed by monitoring <em>/var/log/suricata/eve.log</em>.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>tail -f /var/log/suricata/eve.log </span><span> </span></code></pre> <h3 id="download-some-rules">Download Some Rules</h3> <p>To get the most out of Suricata you will want to download some rules. The Emerging Threats Open rules are freely available and can be installed with the following commands:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd /etc/suricata </span><span>curl http://rules.emergingthreats.net/open/suricata-2.0/emerging.rules.tar.gz | tar zxvf - </span><span>systemctl restart suricata </span><span> </span></code></pre> Another IDS rule downloader - rulecat 2015-05-27T23:23:00+00:00 2015-05-27T23:23:00+00:00 Unknown https://jasonish.org/blog/another-ids-rule-downloader-rulecat/ <div class="alert alert-warning" role="alert"> July 21, 2021 -- This tool is no longer maintained. Instead you should look at Suricata-Update which is the successor to this tool. <a href="https://github.com/OISF/suricata-update">https://github.com/OISF/suricata-update</a> </div> <p>I didn't mean to do it, but I created yet another tool to download IDS rules, with a specific focus on Suricata. Like some other people, I've always used my own scripts for downloading rules instead of using existing tools, and I finally pulled some of that together and dropped it into my <a href="https://github.com/jasonish/py-idstools">idstools</a> project.</p> <h3 id="installation">Installation</h3> <p>With pip (as its written in Python):</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pip install idstools </span></code></pre> <p>From source, <a href="https://github.com/jasonish/py-idstools/releases/download/0.5.0/idstools-0.5.0.tar.gz">https://github.com/jasonish/py-idstools/releases/download/0.5.0/idstools-0.5.0.tar.gz</a>, where idstools-rulecat can be run from the exploded tarball without installing (also works with a git clone).</p> <h3 id="usage">Usage</h3> <p>Probably the simplest usage, assuming you have Suricata already installed is something like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>idstools-rulecat --rules-dir /etc/suricata/rules </span></code></pre> <p>This will download the ET Open rules for your version of Suricata and drop them in /etc/suricata/rules. Other useful output options include:</p> <ul> <li><em>--merged</em> to merge all the rules into a single file making it easier to include into your suricata.yaml.</li> <li><em>--yaml-fragment</em> to dump a fragment of YAML for inclusion in your suricata.yaml that lists each rule file downloaded.</li> </ul> <p>Of course you can also disable, enable and modify rules. To get sample configuration files for doing so, run:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>idstools-rulecat --dump-sample-configs </span></code></pre> <p>which will drop the following files into your current directory:</p> <ul> <li><em>disable.conf</em> - allows disabling of rules by ID or regular expression.</li> <li><em>enable.conf</em> - allows enabling of rules by ID or regular expression.</li> <li><em>modify.conf</em> - rule modifications</li> <li><em>threshold.in</em> - an input threshold.conf that idstools-rulecat will expand (experimental)</li> </ul> <p>As with many Python programs using the <a href="https://docs.python.org/2/library/argparse.html">argparse</a> module, a configuration file can be created by dropping command line arguments in a file and calling <em>idstools-rulecat</em> like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>idstools-rulecat @/path/to/rulecat.conf </span></code></pre> <p>For example, I use a <em>rulecat.conf</em> like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>--suricata=/usr/bin/suricata </span><span>--merged=rules/merged.rules </span><span>--disable=disable.conf </span><span>--enable=enable.conf </span><span>--modify=modify.conf </span><span>--threshold-in=threshold.in </span><span>--threshold-out=etc/threshold.config </span><span>--post-hook=sudo kill -USR2 $(cat /var/run/suricata.pid) </span><span>--etpro=ETPRO_CODE </span></code></pre> <p>Note the <em>--etpro</em> which will ET Pro rules instead of the ET Open rules. For full usage, head over to <a href="http://idstools.readthedocs.org/en/latest/tools/rulecat.html">http://idstools.readthedocs.org/en/latest/tools/rulecat.html</a>. This is a work in progress, and is primarily being developed to satisfy my needs, perhaps it will be useful to others as well.</p> Suricata Beta (2.1beta3) RPMs for EL7 2015-01-30T23:48:32+00:00 2015-01-30T23:48:32+00:00 Unknown https://jasonish.org/blog/suricata-beta-2-1beta3-rpms-for-el7/ <p><strong>If you got here looking for Suricata RPMs, please head over <a href="https://forum.suricata.io/t/guide-suricata-rpms-for-centos-and-fedora/752">https://forum.suricata.io/t/guide-suricata-rpms-for-centos-and-fedora/752</a></strong></p> <p><del>I have created a Suricata Beta yum repository for CentOS/EL 7 to accompany the <a href="http://blog.jasonish.org/2014/12/04/suricata-rpm-for-el-and-centos-7/">stable repo</a> which includes the just released Suricata 2.1beta3. Head to <a href="http://codemonkey.net/suricata-rpms/">http://codemonkey.net/suricata-rpms/</a> for further details on installing the RPMs. Please let me know if you'd like to see the same for Fedora.</del></p> Suricata with EveBox on a Honeypot 2015-01-26T19:22:03+00:00 2015-01-26T19:22:03+00:00 Unknown https://jasonish.org/blog/suricata-wevebox-on-a-honeypot/ <p>I recently installed some honeypot software and am logging the traffic with Suricata into Elastic Search with Logstash. I know its a bit of a risk to expose Elastic Search like this, but I thought it could make a good demo for <a href="https://github.com/jasonish/evebox">EveBox</a>. To check it out head over to <a href="https://demo.evebox.org/">https://demo.evebox.org/</a> with the username "evebox" with the same as the password. This probably won't be up for too long, it will depend on how useful the honeypot is to me at this time.</p> <p>Update - 2017-11-24: Update URL to point to the EveBox demo.</p> <p>Update - 2021-07-11: This demo no longer exists.</p> Suricata RPM for EL 7 Updated to 2.0.5 2014-12-12T17:36:16+00:00 2014-12-12T17:36:16+00:00 Unknown https://jasonish.org/blog/suricata-rpm-for-el-7-updated-to-2-0-5/ <p>I've updated the Suricata RPM for EL 7 to <a href="http://suricata-ids.org/2014/12/12/suricata-2-0-5-available/">2.0.5</a>. If you have the repo configured just yum update, otherwise see this <a href="http://blog.jasonish.org/2014/12/04/suricata-rpm-for-el-and-centos-7/">post</a> for more information.</p> Suricata RPM for EL and CentOS 7 2014-12-05T00:47:12+00:00 2014-12-05T00:47:12+00:00 Unknown https://jasonish.org/blog/suricata-rpm-for-el-and-centos-7/ <p>[2016-07-25: This information is no longer valid, header over to https://codemonkey.net/suricata-rpms/ for up to date links.] I've taken the Suricata package as found in Fedora and rebuilt it for CentOS 7. This should be similar to how the package would exist in EPEL (and hopefully it makes its way there). To get the package with yum, first install the yum repository package (note: you must already have <a href="https://fedoraproject.org/wiki/EPEL">EPEL</a> installed):</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>~~rpm -Uvh http://codemonkey.net/files/rpm/suricata/el7/suricata-release-el-7-1.el7.noarch.rpm~~ </span></code></pre> <p>then install Suricata:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install suricata </span></code></pre> <p><del>To just get the package, head over to <a href="http://codemonkey.net/files/rpm/suricata/el7/">http://codemonkey.net/files/rpm/suricata/el7/</a>.</del></p> Suricata + ELK in Docker 2014-09-12T17:32:14+00:00 2014-09-12T17:32:14+00:00 Unknown https://jasonish.org/blog/suricata-elk-in-docker/ <p>While getting familiar the very popular <a href="http://www.docker.com">Docker</a> Linux container tool, I went against best practice and put Suricata, Logstash, Elastic Search and Kibana into a container that is looking promising for demonstration purposes. If you already run this stack on one machine, it might be suitable for real use as well. What you get is a very simple to run application container that abstracts all the tools above into a single application. Assuming you have Docker already <a href="https://docs.docker.com/installation/">installed</a>, you can get a feel for Suricata + ELK with a couple commands:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>git pull https://github.com/jasonish/docker-suricata-elk.git </span><span>cd docker-suricata-elk </span><span>./launcher start -i eth0 </span><span> </span></code></pre> <p>The first time <em>./launcher start</em> is run, Docker will pull down the container file system layers so it may take a while. Subsequent starts will be much quicker. Once it looks like it is up and running, point your browser at http://localhost:7777. A few notes:</p> <ul> <li>Docker containers are more or less stateless. Changes to the filesystem inside the container are not persisted over a restart. Instead any data that needs to be persisted will end up in the ./data directory where you started the launcher.</li> <li>This container uses host networking instead of the usual isolated network you find with Docker containers. This is to give the container access to your physical interfaces. This alone has me questioning Docker for network monitoring deployments.</li> <li>As host networking is used, the container will probably fail if you have existing applications bound to port 7777 or 9200. Making these ports configurable is on the todo.</li> <li>The containers log directory is available from the host system. Take a look in ./data/log.</li> <li>Suricata is built from git master.</li> <li><code>./launcher enter</code> will give you a shell inside the running container. This is useful to take a look around the runtime environment. Just remember that any changes you make will not be persistent.</li> <li><code>./launcher bash</code> will start a new container with the bash shell and nothing running. This is mostly useul for development.</li> <li>If running a VM, allocate 2GB of memory and/or create a swap file. These are not lightweight applications.</li> </ul> <p>Project links:</p> <ul> <li><a href="https://github.com/jasonish/docker-suricata-elk">https://github.com/jasonish/docker-suricata-elk</a></li> <li><a href="https://registry.hub.docker.com/u/jasonish/suricata-elk/">https://registry.hub.docker.com/u/jasonish/suricata-elk/</a></li> </ul> EveBox - An "eve" Event Viewer for Suricata/ElasticSearch 2014-05-23T00:56:51+00:00 2014-05-23T00:56:51+00:00 Unknown https://jasonish.org/blog/evebox-an-eve-event-viewer-for-suricataelasticsearch/ <p><a href="http://www.elasticsearch.org/overview/kibana/">Kibana</a> is really good for getting a high level overview of your Suricata events, but I didn't find it very useful for reviewing individual events, and I'm not really sure if Kibana is really built around that idea, so I created EveBox, a web based event viewer for Suricata events being logged to Elastic Search in "eve" format with a focus on keyboard navigation: <a href="/images/evebox-inbox-dark-ee33fd8442792fed1e8ad1486325611f.png"><img alt="evebox" src="/images/evebox-inbox-dark-ee33fd8442792fed1e8ad1486325611f.png?w=660" height="303" width="660" /></a></p> <p>Yes, forgive the "yet another bootstrap app" looks, but I'm not a designer nor do I pretend to be. If you log thousands, or even hundreds of events per second, then EveBox is probably not for you, the "inbox" will be unmanageable. However, if you run a highly tuned ruleset, EveBox gives you full keyboard navigation review of those events.</p> <p>Its still a little crude in some areas, for example, if you open an event to get further details you are just going to see the JSON as returned by Elastic Search, personally I like this but I think something a little easier on the eyes is needed.</p> <p>It will also be more useful with eve logs the alert packet, but for now it pivot to <a href="/blog/new-dumpy-release-multiple-spools-and-single-binary-install">Dumpy</a> (a rather basic daemonlogger spool directory frontend) to get a packet capture of the alert triggering data.</p> <p>I've also learned that while Elastic Search is great (well, more like awesome) for searching, its not the best tool for mass updates of records such as "tagging" every entry that matches a query.</p> <p>For such cases it might be useful to introduce a backend at some point so the HTML5 application can hand off some of the grunt work to a backend server that can handle the batch tasks.</p> <p>PostgreSQL 9.4 with its new JSON(b) column could also prove to a very capable data store for Suricata eve events (Cassandra might be another option as well).</p> <p>If you would like to try, go get the latest <a href="https://github.com/jasonish/evebox/releases">release</a> and drop it on a web server.</p> <p>For now its just straight up HTML like Kibana, so its basically a 0 effort install.</p> <p>If that sounds to hard head over to <a href="http://codemonkey.net/evebox/">http://codemonkey.net/evebox</a>, click on settings and enter the URL to your Elastic Search server. The "inbox" won't be there until you configure Logstash accordingly, but you can still review events under "All'.</p> <p>NOTE: My server will not connect to your Elastic Search, the settings only tell the HTML5 application where to connect to Elastic Search).</p> <p>If you are not yet using Suricata, Snort can easily be used instead.</p> <p>For more info on sending Snort events to Elastic Search in "eve" format see my post <a href="/blog/snort-logstash-elastic-search-and-kibana/">Snort, Logstash, Elastic Search and Kibana...</a></p> <p><a href="https://github.com/jasonish/evebox">EveBox on Github</a></p> New Dumpy Release - Multiple Spools and Single Binary Install 2014-05-07T21:01:23+00:00 2014-05-07T21:01:23+00:00 Unknown https://jasonish.org/blog/new-dumpy-release-multiple-spools-and-single-binary-install/ <p>I've made some changes to my simple to install and use PCAP spool web frontend <a href="http://blog.jasonish.org/2012/12/dumpy-a-simple-pcap-spool-file-frontend.html">Dumpy</a> including:</p> <ul> <li>A rewrite in Go, mostly for entertainment purposes, but the really easy to use concurrency, and single binary installation make Go a good choice for small applications like this.</li> <li>Multiple spool directory support.</li> <li>A decoder for translating a Suricata JSON style event to a pcap filter (in additions to the existing "fast" style event decoding).</li> </ul> <p>Check it out over here <a href="https://github.com/jasonish/dumpy">https://github.com/jasonish/dumpy</a>.</p> Snort, Logstash, Elastic Search and Kibana... 2014-04-17T03:39:00+00:00 2014-04-17T03:39:00+00:00 Unknown https://jasonish.org/blog/snort-logstash-elastic-search-and-kibana/ <p>After having fun with Suricata's new eve/json logging format and the Logstash/Elastic Search/Kibana combination (see <a href="https://home.regit.org/2014/01/a-bit-of-logstash-cooking/">this</a> and <a href="http://blog.inliniac.net/2014/03/25/suricata-2-0-and-beyond/">this</a>), I wanted to get my Snort events into Elastic Search as well. Using my <a href="https://pypi.python.org/pypi/idstools">idstools</a> python library I wrote <a href="http://idstools.readthedocs.org/en/0.4.0/u2json.html">u2json</a>, a tool that will process a unified2 spool directory (much like barnyard) and convert the events to Suricata-style JSON.</p> <p>Usage is relatively simple, assuming Snort is logging to /var/log/snort, the following command line should do:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> idstools-u2json -c /etc/snort/snort.conf </span><span> --directory /var/log/snort </span><span> --prefix unified2.log </span><span> --follow --bookmark </span><span> --output /var/log/snort/alerts.json </span><span> </span></code></pre> <p>As the output is in the same format as Suricata's you can refer to this <a href="https://redmine.openinfosecfoundation.org/projects/suricata/wiki/_Logstash_Kibana_and_Suricata_JSON_output">guide</a> for the Logstash setup.</p> <p>One extra step I did was use Logstash to add an "engine" field to each entry. This can be accomplished by adapting the following Logstash configuration:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>input { </span><span> file { </span><span> path =&gt; [&quot;/var/log/suricata/eve.json&quot;] </span><span> codec =&gt; json </span><span> type =&gt; &quot;suricata-json&quot; </span><span> } </span><span> file { </span><span> path =&gt; [&quot;/var/log/snort/alerts.json&quot;] </span><span> codec =&gt; json </span><span> type =&gt; &quot;snort-json&quot; </span><span> } </span><span>} </span><span> </span><span>filter { </span><span> if [type] == &quot;suricata-json&quot; { </span><span> mutate { </span><span> add_field =&gt; { </span><span> &quot;engine&quot; =&gt; &quot;suricata&quot; </span><span> } </span><span> } </span><span> } </span><span> </span><span> if [type] == &quot;snort-json&quot; { </span><span> mutate { </span><span> add_field =&gt; { </span><span> &quot;engine&quot; =&gt; &quot;snort&quot; </span><span> } </span><span> } </span><span> } </span><span>} </span><span> </span></code></pre> <p>Checkout out the <a href="http://idstools.readthedocs.org/en/0.4.0/u2json.html">documentation</a> for information.</p> Easy Unified2 File Reading in Python 2013-07-20T04:04:00+00:00 2013-07-20T04:04:00+00:00 Unknown https://jasonish.org/blog/easy-unified2-file-reading-in-python/ <p>I recently consolidated my Python code bits for dealing with Snort and Suricata unified2 log files into a project called <a href="https://github.com/jasonish/idstools.py">idstools</a>. While I'll be adding more than just unified2 reading support, that is about it for now. While it can be installed with pip (pip install idstools), if you just want to play around with it I suggest cloning the repo (git clone https://github.com/jasonish/idstools.py). You can then use the REPL or write test scripts from within the idstools.py directory without having to install the library (yeah, basic stuff for Python developers). idstools does come with a few example programs that demonstrate unified2 file reading, namely, <a href="https://github.com/jasonish/idstools.py/blob/master/examples/u2fast.py">u2fast.py</a>, <a href="https://github.com/jasonish/idstools.py/blob/master/examples/u2tail.py">u2tail.py</a> and <a href="https://github.com/jasonish/idstools.py/blob/master/examples/u2spewfoo.py">u2spewfoo.py</a> (a simple clone of the Snort provided u2spewfoo).</p> <h3 id="basic-unified2-file-reading">Basic Unified2 File Reading</h3> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>from idstools import unified2 </span><span> </span><span>reader = unified2.FileEventReader(&quot;tests/merged.log&quot;) </span><span>for event in reader: </span><span> print(&quot;Event:n%s&quot; % str(event)) </span><span> </span></code></pre> <p>This few lines of code will iterate through each record in the specified unified2 log files, aggregate the records into events and return each event as a dict. If straight up record is reading is more what you are after then check out <a href="https://idstools.readthedocs.org/en/latest/api-unified2.html#idstools.unified2.FileRecordReader">unified2.FileRecordReader</a>, or the lower level <a href="https://idstools.readthedocs.org/en/latest/api-unified2.html#idstools.unified2.read_record">unified2.read_record</a> function. Each event is represented as a <em>dict</em> containing the fields of a unified2 event record, with the associated packets represented as a list in <em>event["packets"]</em> and extra data records represented as a list in <em>event["extra-data"]</em>.</p> <h3 id="resolving-event-message-and-classification-names">Resolving Event Message and Classification Names</h3> <p>To make event reading just a little more useful, code to map signature and classifications IDs to descriptions is provided.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>from idstools import maps </span><span> </span><span># Create and populate the signature message map. </span><span>sigmap = maps.MsgMap() </span><span>sigmap.load_genmsg_file(&quot;gen-msg.map&quot;) </span><span>sigmap.load_sidmsg_file(&quot;sid-msg.map&quot;) </span><span> </span><span># Get the description for 1:498. </span><span>print(&quot;Message for 1:498: %s&quot; % (sigmap.get(1, 498).msg)) </span><span> </span><span># Create and populate the classification map. </span><span>classmap = maps.ClassificationMap() </span><span>classmap.load_classification_file(&quot;classification.config&quot;) </span><span>print(&quot;The description for classification id 9 is %s, with priority %d.&quot; % ( </span><span> classmap.get(9).description, classmap.get(9).priority)) </span><span> </span></code></pre> <p>The example program <a href="https://github.com/jasonish/py-idstools/blob/0.3.1/idstools/scripts/u2fast.py">u2fast.py</a> is a complete example of reading events from one or more files, resolving event descriptions and classification names and printing the event in a "fast" like style.</p> <h3 id="spool-reading">Spool Reading</h3> <p>idstools also contains a spool reader for processing a unified2 spool directory as commonly used by Snort and Suricata. It supports bookmarking, deleting files, and open and close hooks which can be used to implement custom archiving.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>from idstools import spool </span><span> </span><span>def my_open_hook(reader, filename): </span><span> print(&quot;File %s has been opened.&quot; % (filename)) </span><span> </span><span>def my_close_hook(reader, filename): </span><span> print(&quot;File %s has been closed.&quot; % (filename)) </span><span> </span><span>reader = spool.Unified2EventSpoolReader( </span><span> &quot;/var/log/snort&quot;, &quot;merged.log&quot;, delete_on_close=False, </span><span> bookmark=True, </span><span> open_hook=my_open_hook, </span><span> close_hook=my_close_hook) </span><span> </span><span>for event in reader: </span><span> print(&quot;Read event with generator-id %d, signature-id %d.&quot; % ( </span><span> event[&quot;signature-id&quot;], event[&quot;generator-id&quot;])) </span><span> </span></code></pre> <p>To see a more complete directory spool process, check out the <a href="https://github.com/jasonish/py-idstools/blob/0.3.1/idstools/scripts/u2tail.py">u2tail.py</a> example program.To learn more checkout <a href="https://github.com/jasonish/idstools.py">idstools over at GitHub</a>, <a href="https://pypi.python.org/pypi/idstools">PyPI</a>, or the work-in-progress documentation on <a href="https://idstools.readthedocs.org/en/latest/">Read the Docs</a>.</p> Dumpy - A Simple PCAP Spool File Frontend 2012-12-18T23:33:00+00:00 2012-12-18T23:33:00+00:00 Unknown https://jasonish.org/blog/dumpy-a-simple-pcap-spool-file-frontend/ <p>Sometimes the best way to try out a new framework or language is to apply it to a domain you already know very well, even if it does happen to reinvent the wheel. <a href="http://www.tornadoweb.org/">Tornado</a> and <a href="http://twitter.github.com/bootstrap/">Twitter Bootstrap</a> are two such frameworks I've been meaning to play with for a while now. The result is <a href="http://dumpy/">Dumpy</a>, a web front-end to pcap spool files as created by <a href="http://www.tcpdump.org/">tcpdump</a>, <a href="http://www.snort.org/snort-downloads/additional-downloads">daemonlogger</a>, or <a href="http://netsniff-ng.org/">netsniff-ng</a> with a very simple configuration and user interface:</p> <p><img src="/images/dumpy-screenshot.png" alt="Dumpy screenshot" /></p> <p>Requirements are minimal, Python 2.6 (so it will run on CentOS 6 with little hassle), Tornado and py-bcrypt which are both trivially installed with pip. It provides its own http server with SSL support, and does not require a database. Usage is also simple. Simply enter a pcap filter, or paste in a Snort or Suricata event in "fast" format, choose start and end times (or simply offsets) and hit download. If interested, start a pcap spool (ie: <code>sudo tcpdump -i eth0 -C 1000 -W10 -G 3600 -w /tmp/eth0.log.%Y%m%d.</code>) then check out Dumpy over here <a href="https://github.com/jasonishish/dumpy">https://github.com/jasonish/dumpy</a>.</p> SpringMVC with Embedded Jetty and Thymeleaf 2012-12-13T22:16:00+00:00 2012-12-13T22:16:00+00:00 Unknown https://jasonish.org/blog/springmvc-with-embedded-jetty-and-thymeleaf/ <p>As mentioned in my previous post on using <a href="http://blog.unx.ca/2012/12/springmvc-with-embedded-jetty.html">embedded Jetty with SpringMVC</a>, I was going to look at simplifying the application by using <a href="http://www.thymeleaf.org/">Thymeleaf</a> instead of JSPs as a view technology.</p> <p>Well, it turns out that it is much more straightforward.</p> <p>First we can remove the JSPC plugin from our pom.xml. Second we can completely remove the web.xml file.</p> <p>Bootstrapping Jetty is much simpler now. We do not have to hook into the Jetty startup process with a lifecycle listener, instead we can directly create the dispatcher servlet and add it to a ServletContextHandler like we would any other servlet:</p> <p>From <a href="https://github.com/jasonish/jetty-springmvc-thymeleaf-template/blob/20121213/src/main/java/config/JettyConfiguration.java">JettyConfiguration.java</a>:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>@Bean </span><span>public ServletHolder dispatcherServlet() { </span><span> AnnotationConfigWebApplicationContext ctx = </span><span> new AnnotationConfigWebApplicationContext(); </span><span> ctx.setParent(applicationContext); </span><span> ctx.register(MvcConfiguration.class); </span><span> DispatcherServlet servlet = new DispatcherServlet(ctx); </span><span> ServletHolder holder = new ServletHolder(&quot;dispatcher-servlet&quot;, servlet); </span><span> holder.setInitOrder(1); </span><span> return holder; </span><span>} </span><span> </span><span>@Bean </span><span>public ServletContextHandler servletContext() throws IOException { </span><span> ServletContextHandler handler = new ServletContextHandler(); </span><span> handler.setContextPath(&quot;/&quot;); </span><span> handler.setResourceBase( </span><span> new ClassPathResource(&quot;webapp&quot;).getURI().toString()); </span><span> handler.addServlet(AdminServlet.class, &quot;/metrics/*&quot;); </span><span> handler.addServlet(dispatcherServlet(), &quot;/&quot;); </span><span> return handler; </span><span>} </span><span> </span><span>@Bean(initMethod = &quot;start&quot;, destroyMethod = &quot;stop&quot;) </span><span>public Server jettyServer() throws IOException { </span><span> Server server = new Server(); </span><span> server.setHandler(servletContext()); </span><span> server.setConnectors(jettyConnectors()); </span><span> return server; </span><span>} </span><span> </span></code></pre> <p>This is much more straightforward, everything wired up with Spring without any lifecycle callback hooks.</p> <p>The setup for Spring to render the Thymeleaf views, is a little more complex, but not fussy. What we do is replace:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>@Bean </span><span>public InternalResourceViewResolver configureInternalResourceViewResolver() { </span><span> InternalResourceViewResolver resolver = </span><span> new InternalResourceViewResolver(); </span><span> resolver.setPrefix(&quot;/WEB-INF/views/&quot;); </span><span> resolver.setSuffix(&quot;.jsp&quot;); </span><span> return resolver; </span><span>} </span><span> </span></code></pre> <p>with</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>@Bean </span><span>public ServletContextTemplateResolver thymeleafTemplateResolver() { </span><span> ServletContextTemplateResolver resolver = </span><span> new ServletContextTemplateResolver(); </span><span> resolver.setPrefix(&quot;/WEB-INF/thymeleaf/&quot;); </span><span> resolver.setSuffix(&quot;.html&quot;); </span><span> resolver.setTemplateMode(&quot;HTML5&quot;); </span><span> resolver.setCacheable(true); </span><span> return resolver; </span><span>} </span><span> </span><span>@Bean </span><span>public SpringTemplateEngine thymeleafTemplateEngine() { </span><span> SpringTemplateEngine engine = new SpringTemplateEngine(); </span><span> engine.setTemplateResolver(thymeleafTemplateResolver()); </span><span> return engine; </span><span>} </span><span> </span><span>@Bean </span><span>public ThymeleafViewResolver thymeleafViewResolver() { </span><span> ThymeleafViewResolver resolver = new ThymeleafViewResolver(); </span><span> resolver.setTemplateEngine(thymeleafTemplateEngine()); </span><span> return resolver; </span><span>} </span><span> </span></code></pre> <p>in <a href="https://github.com/jasonish/jetty-springmvc-thymeleaf-template/blob/20121213/src/main/java/config/MvcConfiguration.java">MvcConfiguration.java</a>.</p> <p>Our templates now look a lot more like plain HTML, and will render better when loaded directly in a browser:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>&lt;!DOCTYPE html&gt; </span><span>&lt;html xmlns:th=&quot;http://www.thymeleaf.org&quot;&gt; </span><span>&lt;head&gt; </span><span> &lt;title&gt;Home&lt;/title&gt; </span><span>&lt;/head&gt; </span><span>&lt;body&gt; </span><span>&lt;h1&gt;Hello World!&lt;/h1&gt; </span><span> </span><span>&lt;p&gt;Server time: &lt;span th:text=&quot;${serverTime}&quot;&gt;&lt;/span&gt;&lt;/p&gt; </span><span> </span><span>&lt;p&gt;Here are some items:&lt;/p&gt; </span><span>&lt;ul&gt; </span><span> &lt;li th:each=&quot;item : ${someItems}&quot; th:text=&quot;${item}&quot;&gt;&lt;/li&gt; </span><span>&lt;/ul&gt; </span><span> </span><span>&lt;p&gt;Do we have a message from the dummy service:&lt;/p&gt; </span><span> </span><span>&lt;div th:if=&quot;${dummyService == null}&quot;&gt; </span><span> &lt;p&gt;No, dummy service is null.&lt;/p&gt; </span><span>&lt;/div&gt; </span><span>&lt;div th:if=&quot;${dummyService != null}&quot;&gt; </span><span> &lt;p&gt;Yes: &lt;span th:text=&quot;${dummyService.getMessage()}&quot;&gt;&lt;/span&gt;&lt;/p&gt; </span><span>&lt;/div&gt; </span><span> </span><span>&lt;p&gt;&lt;a href=&quot;resources/static.txt&quot;&gt;A static file.&lt;/a&gt;&lt;/p&gt; </span><span> </span><span>&lt;p&gt;&lt;a href=&quot;metrics&quot;&gt;Yammer Metrics&lt;/a&gt;&lt;/p&gt; </span><span> </span><span>&lt;/body&gt; </span><span>&lt;/html&gt; </span><span> </span></code></pre> <p>With the build process being simple (no dependence on maven plugins) we can switch to a much less verbose <a href="http://www.gradle.org/">Gradle</a> build file:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>apply plugin:&#39;java&#39; </span><span>apply plugin:&#39;application&#39; </span><span> </span><span>version = &#39;0.0.1-SNAPSHOT&#39; </span><span> </span><span>mainClassName = &quot;ca.unx.template.Main&quot; </span><span>applicationName = &quot;jetty-springmvc-thymeleaf-template&quot; </span><span> </span><span>repositories { </span><span> mavenCentral() </span><span>} </span><span> </span><span>dependencies { </span><span> compile(&quot;org.springframework:spring-webmvc:3.1.3.RELEASE&quot;) { </span><span> // Commons-logging excluded in favour of SLF4j. </span><span> exclude module: &#39;commons-logging&#39; </span><span> } </span><span> </span><span> compile &quot;cglib:cglib:2.2.2&quot; </span><span> compile &quot;org.thymeleaf:thymeleaf-spring3:2.0.14&quot; </span><span> compile &quot;org.eclipse.jetty:jetty-webapp:8.1.8.v20121106&quot; </span><span> compile &quot;com.yammer.metrics:metrics-servlet:2.2.0&quot; </span><span> </span><span> /* Logging. */ </span><span> def slf4jVersion = &quot;1.7.1&quot; </span><span> compile &quot;ch.qos.logback:logback-classic:1.0.9&quot; </span><span> compile &quot;org.slf4j:slf4j-api:$slf4jVersion&quot; </span><span> compile &quot;org.slf4j:jcl-over-slf4j:$slf4jVersion&quot; </span><span> compile &quot;org.codehaus.groovy:groovy:1.8.6&quot; </span><span>} </span><span> </span></code></pre> <p>A complete template project can be found <a href="https://github.com/jasonish/jetty-springmvc-thymeleaf-template/tree/20121213">here</a>. Note that this links to tag which is the state of the code at the time of this writing.</p> SpringMVC with Embedded Jetty 2012-12-08T01:46:00+00:00 2012-12-08T01:46:00+00:00 Unknown https://jasonish.org/blog/springmvc-with-embedded-jetty/ <p>As an accidental Java developer I've never been comfortable deploying applications <em>into</em> a container, especially when the web interface is secondary to the primary purpose of the application. Instead I prefer to programatically create and manage the web interface rather than have it manage me. Currently SpringMVC is my Java web framework of choice (due to company convention more than anything else) and it is built around the idea of being managed by a container such as Jetty or Tomcat.</p> <p>While there is no shortage of existing posts on using SpringMVC with embedded Jetty, they fail for me due to the following reasons:</p> <ul> <li>They embed Jetty just enough to bootstrap something that looks like a classic Java web application - not what I want!</li> <li>They don't address JSPs - the default view technology used by SpringMVC.</li> </ul> <h3 id="compile-those-jsps">Compile Those JSPs</h3> <p>By compiling the JSPs prior to deployment we can save ourselves from the hassle of classpath issues, especially relating to tag libs.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>&lt;plugin&gt; </span><span> &lt;groupId&gt;org.mortbay.jetty&lt;/groupId&gt; </span><span> &lt;artifactId&gt;jetty-jspc-maven-plugin&lt;/artifactId&gt; </span><span> &lt;version&gt;${jetty.version}&lt;/version&gt; </span><span> &lt;executions&gt; </span><span> &lt;execution&gt; </span><span> &lt;phase&gt;compile&lt;/phase&gt; </span><span> &lt;id&gt;jspc&lt;/id&gt; </span><span> &lt;goals&gt; </span><span> &lt;goal&gt;jspc&lt;/goal&gt; </span><span> &lt;/goals&gt; </span><span> &lt;configuration&gt; </span><span> &lt;webAppSourceDirectory&gt; </span><span> ${basedir}/src/main/resources/webapp </span><span> &lt;/webAppSourceDirectory&gt; </span><span> &lt;webXml&gt; </span><span> ${basedir}/src/main/resources/webapp/WEB-INF/web.xml </span><span> &lt;/webXml&gt; </span><span> &lt;webXmlFragment&gt; </span><span> ${basedir}/target/classes/webapp/WEB-INF/web.xml-frag </span><span> &lt;/webXmlFragment&gt; </span><span> &lt;/configuration&gt; </span><span> &lt;/execution&gt; </span><span> &lt;/executions&gt; </span><span>&lt;/plugin&gt; </span><span> </span></code></pre> <p>This plugin will compile all the JSPs found under src/main/resources/webapp and modify the existing web.xml to direct requests for the JSPs to the pre-compiled versions. This avoids compiling them at runtime and speeds up the time it takes to respond to the first request for a JSP.</p> <p>You may also notice that the JSPs are under src/main/resources/webapp instead of the more common src/main/webapp. This is to avoid extra lines in the pom.xml to pull in webapp as a resource, as we are packaging as a jar.</p> <h3 id="bootstrapping-jetty">Bootstrapping Jetty</h3> <p>I use Spring annotations to configure Jetty and import its <a href="https://github.com/jasonish/jetty-springmvc-jsp-template/tree/20121207/src/main/java/config/JettyConfiguration.java">@Configuration class</a> into my <a href="https://github.com/jasonish/jetty-springmvc-jsp-template/tree/20121207/src/main/java/config/RootConfiguration.java">root context configuration</a> class. In order for the SpringMVC dispatcher servlet to access beans created in the root context, this class needs to be application context aware:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>@Configuration </span><span>public class JettyConfiguration implements ApplicationContextAware { </span><span> private ApplicationContext applicationContext; </span><span> </span><span> @Override </span><span> public void setApplicationContext(ApplicationContext applicationContext) </span><span> throws BeansException { </span><span> this.applicationContext = applicationContext; </span><span> } </span><span> </span></code></pre> <p>this is used as the parent context for the new context that will be created for the SpringMVC dispatcher servlet:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>@Bean </span><span>public ServletHolder dispatcherServlet() { </span><span> AnnotationConfigWebApplicationContext ctx = </span><span> new AnnotationConfigWebApplicationContext(); </span><span> ctx.setParent(applicationContext); </span><span> ctx.register(MvcConfiguration.class); </span><span> DispatcherServlet servlet = new DispatcherServlet(ctx); </span><span> ServletHolder holder = new ServletHolder(&quot;dispatcher-servlet&quot;, </span><span> servlet); </span><span> holder.setInitOrder(1); </span><span> return holder; </span><span>} </span><span> </span></code></pre> <p>There are still a few more tricks required to get the dispatcher servlet registered without defining it in your web.xml. We must also create a Jetty web application context, this creates the JSP servlet as well as a <em>default</em> servlet as would be created during the instantiation of a classic war based application:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>@Bean </span><span>public WebAppContext jettyWebAppContext() throws IOException { </span><span> WebAppContext ctx = new WebAppContext(); </span><span> ctx.setContextPath(&quot;/&quot;); </span><span> ctx.setWar(new ClassPathResource(&quot;webapp&quot;).getURI().toString()); </span><span> </span><span> /* We can add the Metrics servlet right away. */ </span><span> ctx.addServlet(AdminServlet.class, &quot;/metrics/*&quot;); </span><span> </span><span> return ctx; </span><span>} </span><span> </span></code></pre> <p>Notice that I've added the metrics servlet, but have yet to add the dispatcher servlet. For some reason, adding the dispatcher servlet here causes JSP views to fail. One option is to fall back to defining the dispatcher servlet in your web.xml (which may be the cleanest option), or you can register the dispatcher servlet in the Jetty lifeCycleStarted callback which we'll do here:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>@Bean </span><span>public LifeCycle.Listener lifeCycleStartedListener() { </span><span> return new AbstractLifeCycle.AbstractLifeCycleListener() { </span><span> @Override </span><span> public void lifeCycleStarted(LifeCycle event) { </span><span> try { </span><span> ServletHolder dispatcherServlet = dispatcherServlet(); </span><span> jettyWebAppContext().getServletHandler() </span><span> .addServletWithMapping(dispatcherServlet, &quot;/&quot;); </span><span> dispatcherServlet.start(); </span><span> } catch (Exception e) { </span><span> logger.error( </span><span> &quot;Failed to start Spring MVC dispatcher servlet&quot;, e); </span><span> } </span><span> } </span><span> }; </span><span>} </span><span> </span><span>@Bean(initMethod = &quot;start&quot;, destroyMethod = &quot;stop&quot;) </span><span>public Server jettyServer() throws IOException { </span><span> Server server = new Server(); </span><span> server.setHandler(jettyWebAppContext()); </span><span> </span><span> /* Add a life cycle listener so we can register the SpringMVC dispatcher </span><span> * servlet after the web application context has been started. */ </span><span> server.addLifeCycleListener(lifeCycleStartedListener()); </span><span> </span><span> server.setConnectors(jettyConnectors()); </span><span> return server; </span><span>} </span><span> </span></code></pre> <p>In summary, this is just another variation of James Ward's post on <a href="http://www.jamesward.com/2012/08/13/containerless-spring-mvc">Containerless Spring MVC</a>. The complete code for this project (which I use as a template) can be at <a href="https://github.com/jasonish/jetty-springmvc-jsp-template">https://github.com/jasonish/jetty-springmvc-jsp-template</a>. The code as referenced in this post can be found in the <a href="https://github.com/jasonish/jetty-springmvc-jsp-template/tree/20121207">20121207</a> tag.</p> <p>Next - As I'm not really a fan of JSPs, or the extra hoops required to make this work right I'll probably look at updating the template with <a href="http://www.thymeleaf.org/">Thymeleaf</a> support, which should greatly simplify things.</p> Some NSM type RPMs. 2012-04-13T13:13:00+00:00 2012-04-13T13:13:00+00:00 Unknown https://jasonish.org/blog/some-nsm-type-rpms/ <p><strong>Note: These RPMs no longer exist. The GitHub repo remains for now: <a href="https://github.com/jasonish/nsm-rpms">https://github.com/jasonish/nsm-rpms</a></strong></p> <p>I've always maintained more or less up to date Snort RPMs for RHEL for personal use and have recently added Suricata. As they may be useful for others I have cleaned them up a little and made a YUM repository for EL6 i386 and x86_64. See the <a href="http://nsm-rpms.unx.ca/">http://nsm-rpms.unx.ca/</a> for more info.</p> <p>A few things to note:</p> <ul> <li>These RPMs use a prefix of /opt/nsm to prevent conflict with similar RPMs you may have installed, its a little bit out of the norm for RPMs and I'm open to comments...</li> <li>Snort and Suricata packages will never be automatically upgraded as often upgrading to a new version requires some administration work such as updating your configuration files. To facilitate this the packages have their version as part of the name and "-latest" pseudo-packages are provided which will always install the latest RPM but you will have to "snort-select" or "suricata-select" the new version to make it active. I'll probably have to add some more detailed documentation about this on the wiki.</li> </ul> <p>As I'm also a regular Fedora user I'll probably add Fedora builds at some point as its little effort to me. If Fedora builds would be useful to you please let me know and I may do it sooner than later.</p> Libevent Examples 2011-03-08T15:42:00+00:00 2011-03-08T15:42:00+00:00 Unknown https://jasonish.org/blog/libevent-examples/ <p>Just looked at my web stats for the first time in a long time and I see that one of the most requested resources are my <a href="http://monkey.org/~provos/libevent/">libevent</a> examples, which no longer exist here. The code however does still exist:</p> <p><a href="https://github.com/jasonish/libevent-examples">https://github.com/jasonish/libevent-examples</a></p> Back to the Mac 2010-04-22T10:28:00+00:00 2010-04-22T10:28:00+00:00 Unknown https://jasonish.org/blog/back-to-the-mac/ <p>Recent issues with Linux on a laptop has given me the desire to try working on a Mac again. Even though I have a pretty well supported laptop (Lenovo x61), its the little issues that have begun to bother me which include not great sleep/suspend support, wireless and audio issues and an almost never ending stream of updates. Yes, you Linux people may say it works fine, but if you've ever used a Mac before you'll know its not as good as a could be, by far.</p> <p>I've also found that its really nice to have the real Microsoft Office handy when you swap documents with other non-Linux using people (yeah, OpenOffice doesn't quite cut it when taking turns editing documents) which meant often having a Windows VM running.</p> <p>The other factor that helps the Mac be a viable option is the majority of the software I develop already builds and installs on a Mac, or can do so with very minor modifications. The stuff that doesn't is so tied to a specific environment that it really requires a virtual machine even when running Linux as the host operating system. This really makes the Mac sound like an ideal system.</p> <p>So yesterday while heading out just before dinner to grab some hamburger buns I walked by the local Mac retailer (hey, its about 20 steps from Safeway) and grabbed a high-res 15" MacBook with the Core i5 2.53ghz processor. They weren't able to upgrade the ram to 8GB on the spot but I decided I couldn't wait.</p> <p>Aesthetically its a beautiful machine. The display is about the best computer display I've ever seen. The 3 primary pieces of software I work on compile out of the box. Its also nice to have Microsoft Office running natively. The thing that drove me away from the Mac before was some of the UI behaviours, like Cmd-Tab switching applications, not just windows of applications. But I'm going to give this a true go and may update this blog now and then with issues that I run into as a developer, developing network-level apps primarily for Linux, making the shift to Mac as a development platform.</p> FVWM: Moving a Window to Another Screen 2009-10-07T13:34:00+00:00 2009-10-07T13:34:00+00:00 Unknown https://jasonish.org/blog/fvwm-moving-a-window-to-another-screen/ <p>Probably the most lacking feature in most multi-screen setups is the ability to move a window from one screen to another with a keyboard shortcut. I do this to bring the window I'm currently working on into my centre monitor so its directly in front of me. There are 2 ways to go about this. The first is moving a window to a specific display and the second is to move (shift) the window to the left or right display (which would just be back and forth in a dual monitor setup).</p> <p>For the first case you could define the following function:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># This function will move a window to the X screen provided in the </span><span># first argument. If the window is in the maximized it will be taken </span><span># out of the maximized state before moving. If we don&#39;t do this it </span><span># will be moved back to the original screen when the maximized state </span><span># is toggled. </span><span># </span><span># If you do not want the pointer to be moved to the windows new </span><span># location then comment out the line containing WarpToWindow. </span><span>DestroyFunc MoveWindowToScreen </span><span>AddToFunc MoveWindowToScreen </span><span>\+ I Current (Maximized) Maximize </span><span>\+ I SetEnv PWX $[pointer.wx] </span><span>\+ I Setenv PWY $[pointer.wy] </span><span>\+ I MoveToScreen $[0] </span><span>\+ I WindowId $[w.id] WarpToWindow $[PWX]p $[PWY]p </span></code></pre> <p>And then add the following to your window operations menu:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>\+ &quot;&amp;1 Move to Screen 1&quot; MoveWindowToScreen 2 </span><span>\+ &quot;&amp;2 Move to Screen 2&quot; MoveWindowToScreen 0 </span><span>\+ &quot;&amp;3 Move to Screen 3&quot; MoveWindowToScreen 1 </span></code></pre> <p>Note that the argument you provide to the MoveWindowToScreen function takes the X display number which may not directly map to the order of the screens on your desk.</p> <p>The second method is to move the screen to the left or the right. Or in the case of a dual monitor setup you could just continually move right to have the window flip/flop between your 2 screens.</p> <p>First download the following file and save it to ~/.fvwm/ShiftToScreen.py: <a href="http://ishbits.googlecode.com/svn/trunk/fvwm/ShiftToScreen.py">ShiftToScreen.py</a><br /> You will also need to update variables <em>screenWidth</em> and <em>screens</em> in ShiftToScreen.py as they are currently setup for my configuration of 3 screens with a width of 1920 pixels each.</p> <p>Then define the following function:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># This function will move a window to the left or right screen </span><span># depending on the first argument which should be on of &quot;left&quot; or </span><span># &quot;right&quot;. It depends on the auxiliary python script </span><span># ShiftToScreen.py. </span><span>DestroyFunc ShiftToScreen </span><span>AddToFunc ShiftToScreen </span><span>\+ I Current (Maximized) Maximize </span><span>\+ I SetEnv PWX $[pointer.wx] </span><span>\+ I Setenv PWY $[pointer.wy] </span><span>\+ I PipeRead &quot;python $./ShiftToScreen.py $[0] $[w.x]&quot; </span><span>\+ I WindowId $[w.id] WarpToWindow $[PWX]p $[PWY]p </span></code></pre> <p>You could then add the following to your windows operations menu:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>\+ &quot;&amp;Left Screen&quot; ShiftToScreen left </span><span>\+ &quot;&amp;Right Screen&quot; ShiftToScreen right </span></code></pre> <p>or define some keyboard shortcuts. For example I have bound Shift-Alt-Left and Shift-Alt-Right to shift the window to the left or right screen:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Key Left WTSF12468 SM ShiftToScreen left </span><span>Key Right WTSF12468 SM ShiftToScreen right </span></code></pre> <p>If you are only working with 2 screens you will only need to define one shortcut to get flip/flop like behaviour as shifting left beyond the leftmost screen will wrap the window around to the right most screen.</p> <p>Enjoy.</p> Ideal Multi-Monitor Setup 2009-09-22T14:36:00+00:00 2009-09-22T14:36:00+00:00 Unknown https://jasonish.org/blog/ideal-multi-monitor-setup/ <p>Multi-monitor configurations are great but I think they leave out some usability aspects that could really make life easier for the keyboard user. When I was just using dual displays I really wanted keyboard shortcuts to do the following:</p> <p>* Move a window from one screen to the other.<br /> * Swap what is seen on each display.</p> <p>When I moved to 3 monitors my needs changed a little.</p> <p>* Shift a window to the screen to the left or the right, rotating around.<br /> * Swap a display with the one to the left or the right.<br /> * Shift all displays to the left or the right.</p> <p>This would make it extremely fast to move the window you are currently working on to your primary display, then push it back to where it came from. Or swap the contents of your right screen with your centre screen if you need to shift your work focus for a while.</p> <p>As far as I know this is not possible with Windows, Mac or most Linux configurations. However, I believe using FVWM as your Linux (ok, *nix) window manager will make this all possible, at least my experiments with dual screens have been positive. I'll update this blog with various functions that should make all of the above achievable.</p> Making that Caps-Lock key an extra Control key 2009-03-16T23:05:00+00:00 2009-03-16T23:05:00+00:00 Unknown https://jasonish.org/blog/making-that-caps-lock-key-an-extra-control-key/ <p>Yes, covered everywhere and I've done it enough time I should know it off by heart now... I used to swap caps and control, but now I just make the caps an additional control and leave the original control as is...</p> <p>~/.xmodmap snippet:</p> <p>remove Lock = Caps_Lock<br /> keysym Caps_Lock = Control_L<br /> add Control = Control_L</p> TAILQ Example 2006-08-19T10:43:00+00:00 2006-08-19T10:43:00+00:00 Unknown https://jasonish.org/blog/tailq-example/ <p>I plan to use this blog to post codebits that I find myself sending to people over and over again. The first example is how to use a TAILQ from sys/queue.h found on the BSDs. Many Linux distributions also have sys/queue.h but may be missing certain macros such as TAILQ_FOREACH(). At the very least I hope this enticies some newer C programmer to use queue.h rather than rolling their own linked list. I’ll jump directly to the code as I feel it is more or less self explanatory.</p> <p>The following code can be downloaded <a href="https://github.com/jasonish/tailq-example">here</a>.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#65737e;">/* </span><span style="color:#65737e;"> * TAILQ example program. </span><span style="color:#65737e;"> */ </span><span> </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdlib.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#65737e;">/* </span><span style="color:#65737e;"> * On many OpenBSD/NetBSD/FreeBSD you could include &lt;sys/queue.h&gt;, but </span><span style="color:#65737e;"> * for portability we&#39;ll include the local copy. </span><span style="color:#65737e;"> */ </span><span style="color:#b48ead;">#include </span><span>&quot;</span><span style="color:#a3be8c;">queue.h</span><span>&quot; </span><span> </span><span style="color:#65737e;">/* </span><span style="color:#65737e;"> * This structure defines each item in our tail queue. It must also </span><span style="color:#65737e;"> * contain an item (TAILQ_ENTRY) that points to the next and previous </span><span style="color:#65737e;"> * items in the tail queue. </span><span style="color:#65737e;"> * </span><span style="color:#65737e;"> * For simplicity, we will be creating a list of integers. </span><span style="color:#65737e;"> */ </span><span style="color:#b48ead;">struct </span><span>tailq_entry { </span><span> </span><span style="color:#b48ead;">int</span><span> value; </span><span> </span><span> </span><span style="color:#65737e;">/* </span><span style="color:#65737e;"> * This holds the pointers to the next and previous entries in </span><span style="color:#65737e;"> * the tail queue. </span><span style="color:#65737e;"> */ </span><span> </span><span style="color:#bf616a;">TAILQ_ENTRY</span><span>(tailq_entry) entries; </span><span>}; </span><span> </span><span style="color:#65737e;">/* </span><span style="color:#65737e;"> * Our tail queue requires a head, this is defined using the </span><span style="color:#65737e;"> * TAILQ_HEAD macro. </span><span style="color:#65737e;"> */ </span><span style="color:#bf616a;">TAILQ_HEAD</span><span>(, tailq_entry) my_tailq_head; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>**</span><span style="color:#bf616a;">argv</span><span>) </span><span>{ </span><span> </span><span style="color:#65737e;">/* Define a pointer to an item in the tail queue. */ </span><span> </span><span style="color:#b48ead;">struct</span><span> tailq_entry *item; </span><span> </span><span> </span><span style="color:#65737e;">/* In some cases we have to track a temporary item. */ </span><span> </span><span style="color:#b48ead;">struct</span><span> tailq_entry *tmp_item; </span><span> </span><span> </span><span style="color:#b48ead;">int</span><span> i; </span><span> </span><span> </span><span style="color:#65737e;">/* Initialize the tail queue. */ </span><span> </span><span style="color:#bf616a;">TAILQ_INIT</span><span>(&amp;my_tailq_head); </span><span> </span><span> </span><span style="color:#65737e;">/* Add 10 items to the tailq queue. */ </span><span> </span><span style="color:#b48ead;">for </span><span>(i = </span><span style="color:#d08770;">0</span><span>; i &lt; </span><span style="color:#d08770;">10</span><span>; i++) { </span><span> </span><span style="color:#65737e;">/* </span><span style="color:#65737e;"> * Each item we want to add to the tail queue must be </span><span style="color:#65737e;"> * allocated. </span><span style="color:#65737e;"> */ </span><span> item = </span><span style="color:#96b5b4;">malloc</span><span>(sizeof(*item)); </span><span> </span><span style="color:#b48ead;">if </span><span>(item == </span><span style="color:#d08770;">NULL</span><span>) { </span><span> </span><span style="color:#96b5b4;">perror</span><span>(&quot;</span><span style="color:#a3be8c;">malloc failed</span><span>&quot;); </span><span> </span><span style="color:#96b5b4;">exit</span><span>(EXIT_FAILURE); </span><span> } </span><span> </span><span> </span><span style="color:#65737e;">/* Set the value. */ </span><span> item-&gt;value = i; </span><span> </span><span> </span><span style="color:#65737e;">/* </span><span style="color:#65737e;"> * Add our item to the end of tail queue. The first </span><span style="color:#65737e;"> * argument is a pointer to the head of our tail </span><span style="color:#65737e;"> * queue, the second is the item we want to add, and </span><span style="color:#65737e;"> * the third argument is the name of the struct </span><span style="color:#65737e;"> * variable that points to the next and previous items </span><span style="color:#65737e;"> * in the tail queue. </span><span style="color:#65737e;"> */ </span><span> </span><span style="color:#bf616a;">TAILQ_INSERT_TAIL</span><span>(&amp;my_tailq_head, item, entries); </span><span> } </span><span> </span><span> </span><span style="color:#65737e;">/* Traverse the tail queue forward. */ </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Forward traversal: </span><span>&quot;); </span><span> </span><span style="color:#bf616a;">TAILQ_FOREACH</span><span>(item, &amp;my_tailq_head, entries) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#d08770;">%d </span><span>&quot;, item-&gt;value); </span><span> } </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span> </span><span style="color:#65737e;">/* Insert a new item after the item with value 5. */ </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Adding new item after 5: </span><span>&quot;); </span><span> </span><span style="color:#bf616a;">TAILQ_FOREACH</span><span>(item, &amp;my_tailq_head, entries) { </span><span> </span><span style="color:#b48ead;">if </span><span>(item-&gt;value == </span><span style="color:#d08770;">5</span><span>) { </span><span> </span><span style="color:#b48ead;">struct</span><span> tailq_entry *new_item = </span><span> </span><span style="color:#96b5b4;">malloc</span><span>(sizeof(*new_item)); </span><span> </span><span style="color:#b48ead;">if </span><span>(new_item == </span><span style="color:#d08770;">NULL</span><span>) { </span><span> </span><span style="color:#96b5b4;">perror</span><span>(&quot;</span><span style="color:#a3be8c;">malloc failed</span><span>&quot;); </span><span> </span><span style="color:#96b5b4;">exit</span><span>(EXIT_FAILURE); </span><span> } </span><span> new_item-&gt;value = </span><span style="color:#d08770;">10</span><span>; </span><span> </span><span> </span><span style="color:#bf616a;">TAILQ_INSERT_AFTER</span><span>(&amp;my_tailq_head, item, new_item, </span><span> entries); </span><span> </span><span style="color:#b48ead;">break</span><span>; </span><span> } </span><span> } </span><span> </span><span> </span><span style="color:#65737e;">/* Do another forward traversal to show the newly added item. */ </span><span> </span><span style="color:#bf616a;">TAILQ_FOREACH</span><span>(item, &amp;my_tailq_head, entries) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#d08770;">%d </span><span>&quot;, item-&gt;value); </span><span> } </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span> </span><span style="color:#65737e;">/* </span><span style="color:#65737e;"> * Delete the item with the value 3. </span><span style="color:#65737e;"> * </span><span style="color:#65737e;"> * We can&#39;t use TAILQ_FOREACH here as TAILQ_FOREACH is not </span><span style="color:#65737e;"> * safe against deletions during the traversal. Some variants </span><span style="color:#65737e;"> * of queue.h have TAILQ_FOREACH_MUTABLE or TAILQ_FOREACH_SAFE </span><span style="color:#65737e;"> * which are safe against deletions. </span><span style="color:#65737e;"> */ </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Deleting item with value 3: </span><span>&quot;); </span><span> </span><span style="color:#b48ead;">for </span><span>(item = </span><span style="color:#bf616a;">TAILQ_FIRST</span><span>(&amp;my_tailq_head); item != </span><span style="color:#d08770;">NULL</span><span>; item = tmp_item) </span><span> { </span><span> tmp_item = </span><span style="color:#bf616a;">TAILQ_NEXT</span><span>(item, entries); </span><span> </span><span style="color:#b48ead;">if </span><span>(item-&gt;value == </span><span style="color:#d08770;">3</span><span>) { </span><span> </span><span style="color:#65737e;">/* Remove the item from the tail queue. */ </span><span> </span><span style="color:#bf616a;">TAILQ_REMOVE</span><span>(&amp;my_tailq_head, item, entries); </span><span> </span><span> </span><span style="color:#65737e;">/* Free the item as we don&#39;t need it anymore. */ </span><span> </span><span style="color:#96b5b4;">free</span><span>(item); </span><span> </span><span> </span><span style="color:#b48ead;">break</span><span>; </span><span> } </span><span> } </span><span> </span><span> </span><span style="color:#65737e;">/* Another forward traversal to show that the value 3 is now gone. */ </span><span> </span><span style="color:#bf616a;">TAILQ_FOREACH</span><span>(item, &amp;my_tailq_head, entries) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#d08770;">%d </span><span>&quot;, item-&gt;value); </span><span> } </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span> </span><span style="color:#65737e;">/* Free the entire tail queue. */ </span><span> </span><span style="color:#b48ead;">while </span><span>(item = </span><span style="color:#bf616a;">TAILQ_FIRST</span><span>(&amp;my_tailq_head)) { </span><span> </span><span style="color:#bf616a;">TAILQ_REMOVE</span><span>(&amp;my_tailq_head, item, entries); </span><span> </span><span style="color:#96b5b4;">free</span><span>(item); </span><span> } </span><span> </span><span> </span><span style="color:#65737e;">/* The tail queue should now be empty. */ </span><span> </span><span style="color:#b48ead;">if </span><span>(!</span><span style="color:#bf616a;">TAILQ_EMPTY</span><span>(&amp;my_tailq_head)) </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">tail queue is NOT empty!</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre>