Invariant Properties

I was talking to some people recently about rolling off my current contract without something lined up and realized that I’ve developed a lot of valuable skills that can help people remain sane while between jobs/contracts or while on furlough.

FINANCIAL AND JOB SEARCH

I won’t address this since they’re covered extensively elsewhere. The only advice I can give is the importance of keeping a cash reserve in the future. The standard advice is one month of expenses in savings for every $10k in salary but we can get by with a lot less as long as there’s still a high demand for people with solid tech skills. (If it’s 2001 and we’re in a tech meltdown then even that much saved may not be enough.) Obviously I would prefer to have money coming in but for now the situation isn’t much different from when I take a (unpaid) vacation.

That brings us to:

VACATIONS AND STAYCATIONS

A vacation can be a great idea if the finances work out and you can actually relax. A staycation, or just hanging out at home without focus, is a horrible idea. You have to keep your edge.

That brings us to:

HAVE A PURPOSE

Everything you do should have a purpose. Not a “higher purpose” (unless that’s what you want), but it should be working towards a goal instead of just screwing around. Ideally you will have a mix of short- and mid-term goals. (You should already be working towards long-term goals.)

The goals should be SMART:

• specific
• measurable
• attainable
• relevant
• time-bound

The goals should also be active, not passive.

A short-term goal is something you can do in a day.

• go through closet, toss torn clothes and give anything that doesn’t fit to charity. (you’re not going to wear them anyway.)
• go through attic, toss outdated technical books. (e.g., do I need to keep books on struts 1.1?)
• go through attic, toss old computers.
• paint the spare bedroom.
• clean up garden for fall, plant bulbs for spring.

A bad short-term goal? Watch a Breaking Bad marathon. I have nothing against the show and it’s fine to watch a few episodes at a time but it’s not time-bound and it’s not active.

The key point is that you do them and you move on.

A mid-term goal is something you can do in 6-8 weeks. You’ll hopefully be working long before then but you can usually continue working towards them at a slower pace afterwards.

• follow a 6-week fitness program.
• take a Coursera class.
• learn a new language. (Seven Languages in Seven Weeks: A Pragmatic Guide to Learning Programming Languages, Seven Databases in Seven Weeks: A Guide to Modern Databases and the NoSQL Movement)
• clean the spare bedroom/home office.

You want to avoid having only “fun” goals. Cleaning out the closet and attic are not fun. But the way I see it is that I have four choices:

• clean it out while I’m between contracts or on furlough.
• clean it out on a weekend when I would rather be doing something else.
• clean it out before moving, perhaps to a new job in a different city.
• force somebody I love to clean it out after I’m dead or disabled.

The first is the least-bad choice.

I try to keep a 2:1 mix between “necessary” goals and “fun” goals.

Finally, if you’re between contracts the job search always has top priority. This is what you do after you’ve followed up all leads for the day and are just filling time. Always keep that in mind.

SHAKE IT UP

Shake things up. Pick paper instead of plastic. Pepsi instead of coke.

Okay, more seriously, I find it helpful to do things a little differently to create a mental space between “then” and “now”. I might wear woven shirts instead of polos. Camping pants instead of jeans. However it’s important to always “get dressed” at a reasonable time. See “staycation” above – you might not be going into an office but it’s not a vacation and it’s not a weekend.

BONUS INTERVIEW ADVICE

Again, it’s easy to find countless good advice online. I’m just going to mention one thing that doesn’t get wide coverage.

Work out a few hours before your interview. It will make you look more energetic while simultaneously taking the edge off your nerves. You don’t want to go too far but a solid workout can dramatically improve your presentation.

This article will discuss using Maven Assembly to create artifacts that can be provided to third-party vulnerability assessment sites (e.g., Veracode) for review.

Static Analysis for Bugs vs. Vulnerability Assessments

At this point everyone is aware of findbugs and uses it religiously, right?

Right?

Findbugs uses static analysis to find bugs. More precisely, it uses static analysis to find bugs that can be found by static analysis. For instance I’ve seen a common pattern of

public void foo(Object obj) {
    if (obj != null) {
        obj.doSomething();
    }

    // lots of obscuring code

    obj.doSomethingElse()
}

Should we check for null a second time? Did we need to check the first time? Should we have returned from the ‘if’ clause?

Why do we also need vulnerability assessment?

What is vulnerability assessment? How is it different from bugs?

The key concept is that vulnerable code is superficially bug-free but is still vulnerable to misuse to attack this site or its users.

An example of vulnerable code is using unsanitized user-provided values. Anyone working on the front-end should know the importance of sanitizing these values.

But what happens when user-provided data is passed out of the front-end, e.g., when it’s written to the database? Will everyone who pulls data from the database know that it might contain unsanitized user-provided data? What about malicious data put into the database via SQL injection?

Static analysis for vulnerability assessment looks a lot like static analysis to find bugs, just a lot more through. Whereas findbugs may take 5 minutes to run Veracode may take a few hours!

(Dynamic analysis takes this a step further and runs the tests against a live system. You can do a light version of this using integration tests.)

Artifacts for Vulnerability Assessment

What do we need to provide for vulnerability assessments? The short answer is three things:

• our compiled code (e.g., java or scala)
• our scripted code (e.g., jsp)
• every jar file we depend on, recursively

We don’t need to provide our source code or resources. The compiled code does need to include debug systems so it can give meaningful error messages – knowing only that there’s 19 defects in a library containing 79 classes isn’t very helpful!

A good format is a tarball containing:

• our jar and wars at the top level, sans version number
• our dependencies under “/lib”, with version number

The version numbers are stripped or retained for tracking purposes. Our code has a continuity across multiple runs. Our dependencies can change at any time and don’t have any continuity beyond what’s explicitly indicated in the version numbers.

Our war files should be stripped of embedded jars since they’ll be present under the ‘lib’ directory. “Thick” war files just increase the size of the uploaded artifact.

We can build this with two maven assembly descriptors.

va-war.xml (vulnerability assessment skinny war)

The first assembly creates a stripped down .war file. I don’t want to call it a skinny war since the intended purpose is different but they have a lot in common.

<assembly
    xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation=
       "http://maven.apache.org/plugins/maven/assembly-plugin/assembly/1.1.2
        http://maven.apache.org/xsd/assembly-1.1.2.xsd">

    <id>va-war</id>
    <formats>
        <format>war</format>
    </formats>

    <includeBaseDirectory>false</includeBaseDirectory>

    <fileSets>
        <!-- grab everything except any jars -->
        <fileSet>
            <directory>target/${project.artifactId}-${project.version}</directory>
            <outputDirectory>/</outputDirectory>
            <includes />
            <excludes>
                <exclude>**/*.jar</exclude>
            </excludes>
        </fileSet>
    </fileSets>
</assembly>

You can exclude additional files if you have sensitive information or a lot of large artifacts:

            <excludes>
                <exclude>**/*.jar</exclude>
                <exclude>**/*.jks</exclude>
                <exclude>**/*.p12</exclude>
                <exclude>**/*.jpg</exclude>
                <exclude>**/db.properties</exclude>
            </excludes>

You need to be careful though – you need to include anything that’s scripted, e.g., jsp files or velocity templates.

va-artifact.xml (vulnerability assessment artifact)

The second artifact collects all of the dependencies and stripped down wars into a single tarball. Our jars and wars are at the top level of the tarball, all dependencies are in a ‘lib’ directory. This makes it easy to distinguish between our artifacts and our dependencies.

<assembly
    xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation=
       "http://maven.apache.org/plugins/maven/assembly-plugin/assembly/1.1.2
        http://maven.apache.org/xsd/assembly-1.1.2.xsd">

    <id>va-artifact</id>
    <formats>
        <format>tar.gz</format>
    </formats>

    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>

        <!-- ******************************************* -->
        <!-- Our code should not include version numbers -->
        <!-- ******************************************* -->
        <dependencySet>
            <includes>
                <include>${project.groupId}:*:jar</include>
                <include>${project.groupId}:*:va-war</include>

                <!-- we could also include subprojects -->
                <include>${project.groupId}.**:*:jar</include>
            </includes>

            <!-- we might have sensitive resources -->
            <excludes>
                <exclude>${project.groupId}:*-properties</exclude>
            <excludes>
            <outputFileNameMapping>${artifact.artifactId}${dashClassifier?}.${artifact.extension}</outputFileNameMapping>
        </dependencySet>

        <!-- *********************************************** -->
        <!-- Our dependencies should include version numbers -->
        <!-- *********************************************** -->
        <dependencySet>
            <outputDirectory>lib</outputDirectory>
            <includes />

            <excludes>
                <exclude>${project.groupId}:*</exclude>
                <exclude>*.pom</exclude>

                <!-- exclude standard APIs -->
                <exclude>javax.*:*</exclude>
                <exclude>dom4j:*</exclude>
                <exclude>jaxen:*</exclude>
                <exclude>jdom:*</exclude>
                <exclude>xml-apis:*</exclude>
            </excludes>
        </dependencySet>
    </dependencySets>
</assembly>

Building the Artifacts

The assembly descriptors are only half of the story. We still need to call maven assembly and we do not want to do it for every build.

This is an ideal time for profiles – we will only build artifacts when a specific profile is specified.

pom.xml for war modules

The necessary addition to the pom.xml file for war modules is modest. We need to call our assembly descriptor but we don’t need to explicitly add dependencies.

<profiles>
    <profile>
        <id>vulnerability-assessment</id>
        <build>
            <plugins>
                <plugin>
                    <artifactId>maven-assembly-plugin</artifactId>
                    <configuration>
                        <descriptors>
                            <descriptor>src/main/assembly/va-war.xml</descriptor>
                        </descriptors>
                    </configuration>
                    <executions>
                        <execution>
                            <id>make-assembly</id>
                            <phase>package</phase>
                            <goals>
                                <goal>single</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

pom.xml for top-level modules

The necessary addition to the pom.xml file for the top-level module is more complex, especially when the distribution assembly is created in a submodule instead of the root module. In this case we need to explicitly add a dependency on both pom files and lite war files. If we don’t specify the former we’ll lose most dependencies, if we don’t specify the latter we’ll lose the .war files.

<profiles>
    <profile>
        <id>vulnerability-assessment</id>
        <build>
            <plugins>
                <plugin>
                    <artifactId>maven-assembly-plugin</artifactId>
                    <configuration>
                        <descriptors>
                            <descriptor>src/main/assembly/va-artifact.xml</descriptor>
                        </descriptors>
                    </configuration>
                    <executions>
                        <execution>
                            <id>make-assembly</id>
                            <phase>package</phase>
                            <goals>
                                <goal>single</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>

        <dependencies>
            <!-- specify parent pom -->
            <dependency>
                <groupId>${project.groupId}</groupId>
                <artifactId>parent</artifactId> <!-- FIXME -->
                <version>${project.version}</version>
                <type>pom</type>
            </dependency>

            <!-- specify each war file and corresponding pom file -->
            <dependency>
                <groupId>${project.groupId}</groupId>
                <artifactId>webapp-1</artifactId> <!-- FIXME -->
                <version>${project.version}</version>
                <type>war</type>
                <classifier>va-war</classifier>
            </dependency>
            <dependency>
                <groupId>${project.groupId}</groupId>
                <artifactId>webapp-1</artifactId> <!-- FIXME -->
                <version>${project.version}</version>
                <type>pom</type>
            </dependency>

            <!-- second... -->
            <dependency>
                <groupId>${project.groupId}</groupId>
                <artifactId>webapp-2</artifactId> <!-- FIXME -->
                <version>${project.version}</version>
                <type>war</type>
                <classifier>va-war</classifier>
            </dependency>
            <dependency>
                <groupId>${project.groupId}</groupId>
                <artifactId>webapp-2</artifactId> <!-- FIXME -->
                <version>${project.version}</version>
                <type>pom</type>
            </dependency>

            <!-- and so on... -->

        </dependencies>
    </profile>
</profiles>

One small gotcha!

There is one small gotcha! in this specific approach. It is possible that the individual web modules will have dependencies on different versions of common libraries. Nobody wants this but once projects reach a certain size you can’t afford the time and effort required to keep all of the modules in sync.

This information will be lost when we do dependency resolution at a common location.

I don’t consider this a problem for two reasons. First, we can perform vulnerability assessments at a finer granularity – essentially perform the analysis at the .war level instead of the .ear file. This guarantees the libraries will match but will tremendously increase our work load if we have a large number of web modules.

Second, our primary focus is the vulnerabilities in our code, not in specific versions of third-party libraries. Those libraries provide important hints to the assessment tools but we only want a full analysis of our code. We can always run separate assessments of the libraries we depend upon if it’s necessary.

Jenkins Veracode Plugin

Finally I want to point out that there’s a Jenkins plugin for Veracode analysis: Veracode Scanner Plugin. It can be used to schedule scans on a regular basis so you don’t find hundreds of defects when you finally remember to run a scan just days before a release.

Prior to the 2012 elections a number of Republican candidates discussed the possibility of a federal shutdown. They were gleeful at the prospect. At least one explicitly stated it would be an opportunity to show the American people how unnecessary most of the government is.

Uh, no.

I’ll never forget a related conversation I had years ago with somebody far wiser than me. He told me to consider stores that sell suits. Think “Mens Warehouse” or “Joseph A. Banks”. It is (largely) not essential to buy a suit on any particular day. There’s no harm in waiting a few days, or even a few weeks. Some men never buy a suit.

BUT, he continued, most men do buy suits. We might not need suits at an abstract philosophical level but in this country at this time most men will face negative consequences if they don’t have one for appropriate situations.

Why did I say “largely”? Sometimes a suit is essential. You suddenly landed an job interview but you either don’t have a suit or it no longer fits. Or, God forbid, you need a black one for your father’s funeral. This is the case for most “necessary but not essential” goods and services – for most people for most of the time there’s some flexibility. But for some people at some times it’s critical.

Still not convinced? You think suits are a symbol of “The Man” and you refuse to wear one? Okay, what about hair cuts? Shoes? A new or used car? Heck, groceries? There are plenty of things you never need – sugary drinks, movies, etc – but most of the economy serves a necessary purpose.

So the current government shutdown?

Nobody “needs” a passport. Unless a close family member is suddenly in the hospital (illness or accident) and they need to travel to them. Again, worst case, parents or children could miss the final days of their child’s or parent’s life.

Nobody “needs” an SBA loan. Unless it’s a business that’s trying to take advantage of an opportunity. Maybe a $2 million dollar company today is a $2 million dollar company in 5 years instead of a $5 million dollar company employing 100 more people. Or, worse case, it will mean that the company was far along the loan process but the shutdown will last long enough that it defaults on a contract. Now the $2 million dollar company isn’t a $5 million company, it’s not a $2 million company, it’s a defunct company that closed years ago.

In a recent post “Popehat” mentioned that a federal criminal case he was prosecuting was dismissed during the last shutdown. We can argue that the judge was an idiot but the bottom line is that somebody avoided prosecution as a direct result of the shutdown. (It’s ironic that he’s largely libertarian and probably won’t agree with my position!)

On and on and on.

That’s the difference between “essential” and “necessary”. You need the former continuously – we can’t send all prison guards home for a week – but that doesn’t mean you never need the latter. It just means that the need is intermittent.

It’s popular in some circles to insist that the government still has a lot of fluff. Ask them what they would cut. People might have plenty of ideas but 1) they’re often based on ignorance of the purpose served and 2) they’re fluff for that person but not for the people they serve or 3) they’re minor programs that have almost no impact on the bottom line.

An notorious example of the first point was Gov. Bobby Jindal’s Republican Response to the President after one of President Obama’s State of the Union addresses. He mentioned unnecessary services like “a volcano observatory, whatever that is”.

A few months later trans-Pacific flights had to be diverted over Alaska due to volcanic eruptions. The jagged bits of rock ash thrown into the air will quickly destroy aircraft.

The less generous among us would observe that people in the Pacific Northwest don’t have much need for the hurricane trackers either. We would never expect to hear the governor of the great State of Washington to ask why we pay for aircraft to fly through hurricanes. Doesn’t anyone realize this is dangerous?

An example of the second point appeared in the late Rocky Mountain News. First was an article on the “dart snail” (or “snail darter”?) and the damage it was causing in the Midwest and how researchers were frantically trying to find a way to stop its spread. It wasn’t just the hundreds of millions of dollars in damage to shipping, it was the problem it was causing to water supplies due to the snails blocking water intake pipes. This research was very necessary before it became critical as cities lost their water supplies entirely.

A month later was a local op-ed about government waste. One of the obvious boondoggles was research into the breeding patterns of a tiny snail in the Midwest. Why would anyone other the egghead scientists care?

Uh, hundreds of millions of dollars in damage to shipping and cities left without water supplies? Isn’t that enough motivation? If not, what will it take to get your attention?

(I don’t know if they published my letter to the editor.)

P.S., what about privatization? If it’s done on behalf of the government then it should be considered part of the government even if the money goes to a third party for third-party employees. However there’s one big difference – that contractor demands a healthy profit. The government does not. If the contractor is truly less expensive than the government then where are the savings coming from?