Category Archives: Uncategorized

A moment of clarity with Puppet and Foreman (Part I)

Over the past four years we’ve deployed a puppet/foreman environment to support Ubuntu 12.04 and 14.04 for our research and production Linux systems. As 12 is approaching end of life and there are no longer Foreman updates available we decided it was a good time to revisit our overall puppet/foreman integration. Over the years it had slowly grown to include a bit of cruft and needed a good haircut. In addition, during the past four years, Foreman had added additional features which made it a good time revisit how the two communicate and where the hand-off in responsibility lie. So with that introduction, the environment we deployed had the following goals and challenges in mind:

  • Clearly define the hand-off between Foreman and Puppet.
    • Profiles and roles vs smart parameters vs host groups vs config groups.
  • Create a clear development to production workflow.
    • Module updating, development, testing.
    • How do we support operational testing (ie, patching, etc) vs longer term development.
  • Improve git integration (yeah, part of above, but a major motivation in of itself)
    • How can we support multiple development efforts?

Final Environment

Skipping ahead to the end, our final Foreman environment consists of the following. We ended up creating a separate dev and production environment which were 100% independent, yet mirrors of each other. We developed a workflow for allowing each developer to have their own environment and setup a clean separation between this development environment and the testing environment/production environment. This allows the production side to quickly test and apply security and other upstream updates without impacting longer term development efforts.

Foreman and Puppet

  • Two Foreman instances, dev and production, each on its own subnet.
  • Only profile classes and their parameters are exposed to foreman.
  • Configuration groups within foreman are used to create roles.
  • Multiple base host groups, one for each subnet/authentication domain.
    • Base host group handles core auth/patching.
    • Second level host groups create services (ie, HPC Cluster Node) and have configuration groups applied, and contain all hosts.
  • Hosts/services are created in pairs, a patch-testing and production both on the production environment.
    • Each production has a corresponding test host which mirrors production and is used to test patches and other minor updates.
    • The testing host also serves as a recovery point, meaning that a nightly backup can quickly be applied and it can be moved into a production role.
  • Development VM’s are attached to the dev foreman server and are short lived hosts for developing and debugging puppet configurations.

Git and Environments

  • Single git repo for puppet all puppet modules and profiles.
  • Git branches mostly correspond to puppet environments
    • ie: development == /etc/code/pupplabs/code/environments/development
    • production – production environment on production foreman server
    • development – set as ‘production’ on development foreman server, development on production foreman.
    • Each developer/sysadmin has their own environment on the dev server, within their environment they can switch to the appropriate branch or use their own developer forks
  • Code workflow:
    1. Developer or sysadmin works in own branch.
    2. Pull request to development branch and import on dev server.
    3. Review and testing in development (if substantial chages, additional step of verifying on production server development environment)
    4. Pull request to production and set parameters.

In parts two and three, I’ll cover a bit more about the motivation and detail behind this setup.

Some additional reading:

Going along on a phishing trip

Looks like the password phishers are finally starting to learn proper grammar and piece together something kinda convincing. Here’s a breakdown on one that I had reported to me over the UMD holiday break. It’s notable for a few reasons:

  • Timing – it was sent over holiday break when lots of academics will be working, but normal administrative/IT staff is off.
  • ‘Realness’ , from copies of UMD’s actual page to references to actual IT help email addresses and phone numbers it passes the sniff test.
  • Attention to detail – Lots of the domain names, etc are put together in a way that won’t raise an alarm to most folks.

Step 1, The email

Here’s the actual email received from these guys. A few things they got correct:

  • The signature information (sans Access & Delivery Services department) is all real and correct.
  • The name of UMD’s IT helpdesk and the included email is correct.
  • Most of the display part of the URL is correct and UMD does have a CAS sitting at /cas/login with the obvious switching of lib and edu.
From: [] On Behalf Of University of Maryland
Sent: Friday, January 1, 2016 9:29 AM
Subject: Library Services
Dear User,

Your access to your library account is expiring soon, and you will be not eligible for Document Delivery Service. To continue to have access to the library services, you must reactivate your account. For this purpose, click the web address below or copy and paste it into your web browser. A successful login will activate your account and you will be redirected to the library homepage.
If you are unable to log in, please contact the IT Service Center at for immediate assistance.

Kind Regards,
Access & Delivery Services
University of Maryland Libraries

McKeldin Library, College Park, MD 20742

Phone: 301-405-0800

Step 2, the URL

The underlying URL in this case points to which in reality is just a 302/redirect to another domain,
$ curl -v ‘’
* Hostname was NOT found in DNS cache
* Trying…
* Connected to ( port 80 (#0)
> GET /activation/access/link.php?M=11158&N=40&L=11&F=H HTTP/1.1
> User-Agent: curl/7.35.0
> Host:
> Accept: */*
< HTTP/1.1 302 Moved Temporarily
< Date: Fri, 01 Jan 2016 21:00:50 GMT
* Server Apache is not blacklisted
< Server: Apache
< X-Powered-By: PHP/5.5.26
< Location:
< Content-Length: 0
< Content-Type: text/html
* Connection #0 to host left intact
Taking a look at the hostnames involved, it appears both of these come from the same, provider.
$ host has address mail is handled by 10
$ host has address mail is handled by 0
It looks like the root, site is used to generate the emails as well based on what’s publicly available.
Screenshot from 2016-01-01 16:08:01 Screenshot from 2016-01-01 16:07:54

Step 3, The login

The login they created for this account is a pretty convincing copy of UMD’s actual CAS login page. The top/forged one uses graphics from UMD. Looking at the source, the login form has been modified to send the response to save.php.

Forged Login page
Forged Login page
Actual UMD Login page
Actual UMD Login page

If you go to the root domain,, there are a half a dozen other universities listed with what I’m assuming are forged copies of their login pages. Entering any username and password into the password field results in a message saying your services have been activated and a link back to UMD’s library main page.

Overall, I’d have to give this one a B+ for the realness factor. Sadly, it probably picked up quite a few accounts given timing, etc.


Building a (Relatively) Affordable Conference Room

(Hardware updated 10/2016)

Our standard Adobe Connect, Skype and Lync compatible conference configuration is designed to provide skype-quality audio and video in and out of a conference room. While a more advanced, high end system would be nice, the types of hardware/software that our ever changing endpoints make that prohibitively expensive. Instead, we rely on most current software’s ability to provide decent full duplex built-in echo cancellation. Flash, Skype and Lync all do this pretty well, we’ve had some difficulty with Webex and early releases of Google Hangouts.

Our goal is to provide complete audio coverage for any participant sitting at a table in our conference room. As our meetings tend to be mostly round-table style discussion a rule of one microphone for every two people allows us to pick up normal conversation-level speech.

Our requirements are that we allow remote participants virtually join meeting in conference rooms ranging in size from 8 through 24 people. Realistically for groups larger than 16-18 the logistics of ensuring that remote participants are fully included in a meeting starts to break down. Distance to TV, etc start to have an detrimental effect on the ability of remote participants to be heavily engaged in a meeting.




Misc Parts:

  • Velcro straps and carpet cover
  • Under-table clips for microphone cables
  • wire wrap

Total Cost (no PC): ~$3,000 $3,700 (8 person) – $5,150 $5,850(24 person)

Accessing the ACE IMS via python

Using hashlib, and binascii we can use python to both generate a digest and grab an ace token for that digest.

While this example only sends one request in a call, you should batch your requests prior to requesting a token. Just send a ‘list’ of tokenRequest objects to requestTokensImmediate. The IMS will support up to 10,000 requests per call.

import hashlib
import binascii
from suds.client import Client


digFile = open(filename,'rb')
hashAlg = hashlib.sha256()
filedigest = binascii.b2a_hex(hashAlg.digest())

client = Client(url)

print  filename, ' ', filedigest

request = client.factory.create('tokenRequest')
request.hashValue = filedigest = filename

result = client.service.requestTokensImmediate('SHA-256-0',request)
print result

The result spits back a token that you can use to validate a file.

[python] [toaster@loach ace-cli]$ python   164182eef9792e2e1c5005cd9240ff508aef042b8fa344597431eae39370c784
  digestService = "SHA-256"
  name = ""
  proofElements[] =
        hashes[] =
        index = 0
        hashes[] =
        index = 0
        hashes[] =
        index = 0
        hashes[] =
        index = 1
  roundId = 2893078
  statusCode = 100
  timestamp = 2011-01-07 13:08:27.000253
  tokenClassName = "SHA-256-0"

Simple Effect

Pivot nicely provides a few nice effects for items. This is useful for images, etc but if you want to make a simple text-based header it needs the tiniest bit of tweaking.

What we need is to translate the reflection so it overlaps the text a little.

We can just modify the Reflection effect and two properties for x & y reflection translation and tweak where the reflection is drawn. Here’s the modified ConfigurableReflection class

public class ConfigurableReflection extends ReflectionDecorator {

    private Component component = null;
    private Graphics2D graphics = null;
    private BufferedImage componentImage = null;
    private Graphics2D componentImageGraphics = null;

    private int yTranslate = 0;
    private int xTranslate = 0;

    public void setXTranslate(int xTranslate) {
        this.xTranslate = xTranslate;

    public void setYTranslate(int yTranslate) {
        this.yTranslate = yTranslate;

    public int getXTranslate() {
        return xTranslate;

    public int getYTranslate() {
        return yTranslate;


    public Graphics2D prepare(Component component, Graphics2D graphics) {
        this.component = component; = graphics;

        int width = component.getWidth();
        int height = component.getHeight();

        componentImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
        componentImageGraphics = componentImage.createGraphics();

        // Clear the image background
        componentImageGraphics.fillRect(0, 0, componentImage.getWidth(), componentImage.getHeight());


        return componentImageGraphics;

    public void update() {
        // Draw the component
        graphics.drawImage(componentImage, 0, 0, null);

        // Draw the reflection
        int width = componentImage.getWidth();
        int height = componentImage.getHeight();

        GradientPaint mask = new GradientPaint(0, height / 4f, new Color(1.0f, 1.0f, 1.0f, 0.0f),
                0, height, new Color(1.0f, 1.0f, 1.0f, 0.5f));

        componentImageGraphics.fillRect(0, 0, width, height);

        componentImageGraphics = null;



        graphics.drawImage(componentImage, 0, 0, null);

        componentImage = null;
        component = null;
        graphics = null;

    public Bounds getBounds(Component component) {
        // MODIFICATION for new translation
        return new Bounds(0, 0, component.getWidth() + xTranslate, component.getHeight() * 2 + yTranslate);

    public AffineTransform getTransform(Component component) {
        AffineTransform transform = AffineTransform.getScaleInstance(1.0, -1.0);
        // MODIFICATION for new translation
        transform.translate(xTranslate, -((component.getHeight() * 2) + yTranslate));

        return transform;

Simple netbeans Pivot module

I got a little tired re-running my entire app just to see what the current pivot layout would look like, so here’s a quick and dirty plugin for Netbeans 6.8 which does the following:

  • Pivot 1.5.1 library
  • XML-editing and formatting for .wtkx files
  • right-click, ‘Preview WTKX’ function


To use this module on a pivot project, Right-click on project, select properties and add the new Pivot 1.5.1 library to your project. When viewing a file, you first need to build your project then right click on the .wtkx files to preview.

Extracting selected pivot tree items

The pivot TreeView only returns selected paths. A path is a list of list indexes which refer to the underlying data model of a tree. Unfortunately, the treeview doesn’t have a convenience method for extracting the items referenced by those paths. Here’s that missing method:

public static List extractSelectedObjects(TreeView tv)
        List retList = new ArrayList();
        for (Path p : tv.getSelectedPaths())
            Object currBranch = tv.getTreeData();

            for (int i : p)
                currBranch = ((List) currBranch).get(i);

        return retList;

Preventing grizzly timeouts

In grizzly, the default timeout for an http process is 5 minutes. This applies to the total activity time. In most cases, this is fine, however for long uploads you may need to extend this. Oh say for uploading 50g files. There is a switch for disabling timeouts, however this is global. What we need is a timeout for connections that are still active. Inactive connections should close 5m after last activity.

Grizzly logs the following:

Aug 12, 2010 3:17:25 PM com.sun.grizzly.http.KeepAliveThreadAttachment timedOut
WARNING: GRIZZLY0023: Interrupting idle Thread: Grizzly-8080(2).
Aug 12, 2010 3:17:27 PM com.sun.grizzly.http.KeepAliveThreadAttachment timedOut
WARNING: GRIZZLY0023: Interrupting idle Thread: Grizzly-8080(3).

The somewhat messy solution is the pick apart the thread that your request is running in and toggle timeouts as needed.

while ((read = != -1)
                if ((System.currentTimeMillis() - lastXmit) > 5000)
                    HttpWorkerThread thread = (HttpWorkerThread)Thread.currentThread();
                    ((SelectionKeyAttachment) thread.getProcessorTask().getSelectionKey().attachment()).setTimeout(lastXmit);
                    lastXmit = System.currentTimeMillis();