ppenzin

Kill The Doctor

Tue, 27 Feb 2018 22:29:19 +0000

LLVM has “Kill the doctor” test utility for disabling DrWatson when tests are running (a scenario would be that a test would throw an exception and DrWatson UI would pop up, freezing the build). Its file comment is worth reproducing in full:

//===- KillTheDoctor - Prevent Dr. Watson from stopping tests ---*- C++ -*-===//
//
//                     The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
// This program provides an extremely hacky way to stop Dr. Watson from starting
// due to unhandled exceptions in child processes.
//
// This simply starts the program named in the first positional argument with
// the arguments following it under a debugger. All this debugger does is catch
// any unhandled exceptions thrown in the child process and close the program
// (and hopefully tells someone about it).
//
// This also provides another really hacky method to prevent assert dialog boxes
// from popping up. When --no-user32 is passed, if any process loads user32.dll,
// we assume it is trying to call MessageBoxEx and terminate it. The proper way
// to do this would be to actually set a break point, but there's quite a bit
// of code involved to get the address of MessageBoxEx in the remote process's
// address space due to Address space layout randomization (ASLR). This can be
// added if it's ever actually needed.
//
// If the subprocess exits for any reason other than successful termination, -1
// is returned. If the process exits normally the value it returned is returned.
//
// I hate Windows.
//
//===----------------------------------------------------------------------===//

What do programmers do

Tue, 07 Jun 2016 22:47:17 +0000

This old rant came up in a conversation with a colleague, here is a link or two. Warren Harrison (former PSU CS chair and IEEE Software editor) wrote an excellent take on what’s wrong with CS education. He does a little bit of old man rambling; and, of course, it is the students, but his article really does expose a lot of problems in the system itself.

A colleague’s reaction was: “this is hopeless, if people draw information about their future field from movies then nothing could be done.” While the idea that undergrads would be inspired by popular culture to choose their major sounds silly, there nothing too astonishing about it. I think the main reason this happens is information vacuum surrounding STEM disciplines in the US. While everyone thinks Computer Science pays well (and therefore you should chose major in it), but no one really knows what is it about; therefore there is such a stark contrast between what people think we do and what we actually do. A separate problem is what is taught in CS classes (the primary reason students stay ignorant to the nature of their field for so long), but that is a big enough topic for a separate post.

While Warren hits it right on about team players and ability to express thoughts in English, Computer Science and especially “algorithm-heavy” parts of it should be viewed as something close to high-level finance industry: there are things there that have drifted pretty far away from “everyday” reality and require a lot of special knowledge to understand.

Back to Plan 9

Thu, 19 Nov 2015 17:50:54 +0000

Some time ago I was not able to resist a hoarding urge and “rescued” an old HP laptop. Pavilion zv5000 (in case you are curious).

I tried a few Linux and BSD incarnations but nothing stuck, primarily due to lack of drivers. And finally I got lucky with Plan 9.

Plan 9 was my first introduction to Unix-like operating systems. I discovered it by accident and was so fascinated with it, that I kept on reading old articles and documentation for it even though I was not really able to put it to much use.

And now, after more than ten years, I suddenly have a laptop and a handful of virtual machines running it. I have to say that prior experience with Unix-like environments is required to get anywhere with it, regardless of what 9fans mailing list has to say.

Run your family like a software company

Sun, 25 Jan 2015 19:52:01 +0000

Responding to change over following a plan

provided a detailed plan is in place to respond to the change, and it is followed precisely

– Half-arsed agile manifesto

For all who are familiar with Agile (but beware, you won’t be able to unsee it):

This is a TED video, it is a couple years old and it is somewhat hillarious. It could have been a parody on Agile Coaching and family counseling, but, unfortunately, he is serious.

He claims that he is using agile ~~to manage his family~~ in his family, but the stuff he is saying sounds more like Agile’s evil corporate cousin with “the simple new process that will make your company more successful”.

There is just so much wrong with the video, from the belief that software development methodologies can be applied to family life, to what Agile and Waterfall are, to fictional “Agile Programming”. Ironicaly, it is not uncommon to see similar things in companies that claim to be doing Agile. On the upside, at least he is not referring to his children as “resources”.

Do not use scripting languages for building your project

Tue, 13 Jan 2015 18:16:37 +0000

Every now and then I come across projects that use some sort of scripting solutions to hold their builds together. What I mean is build scripts written in Bash, Python, Perl or any other general-purpose scripting language. Usually scripts are used as wrappers for makefiles or any other build tools and are trying to make up for functionality those tools are lacking, though there are those rare cases when they are used to invoke the compiler directly. This has to stop and here is why:

Is is wasteful. You are requiring people to carry around a shell or an interpreter (often legacy) just for the sake of running a handful of scripts (often just a single script). Unless you are using those things for what they are supposed to be used, that is an unnecessary dependency. Moreover, building is fairly repetitive and tedious by nature, so you are looking at chasing a lot of little bugs that cause substantial trouble.
It is not quite cross-platform. This is mostly true for Bash idioms and other linuxisms. Not everyone in the world has the same set up as you do, nor they need to have.
It does not do you any good personally, unless you want to do program in scripting languages for a living. It is a much better investment of time to learn to do advanced stuff with popular build tools than hone your skills in languages that you would never use.

Here are some suggestions:

Switch to more advanced build tools. CMake and Autotools help when Makefile-based projects get out of hand; Maven is a powerful tool for Java projects, and so on. Those tools have been around for a while and have become de facto standards. They have the common use-cases implemented and are more consistent that hand-rolled build systems.
Write your own build modules: use extendable build tools (CMake for example) and write extensions for it. That way your build scripts are kept in a language designed for builds.
Create your own build tool. If nothing helps, try to implement the custom functionality that you need as a complete product rather than set of scripts. Designing it will help you review your build process and might trigger some positive changes for the project. And if it succeeds you get another marketable app almost for free.

FreeBSD: QuickCheck (tf-random) and `invalid byte sequence'

Wed, 25 Jun 2014 00:46:12 +0000

Have you ran into this while installing QuickCheck on FreeBSD?

QuickCheck-2.7.5 depends on tf-random-0.5 which failed to install.
tf-random-0.5 failed during the final install step. The exception was:
/tmp/pkgConf-tf-random-043046.5: hGetContents: invalid argument (invalid byte
sequence)

I wasted a lot of time trying to debug this, but today I was fighting with a similar problem with Jekyll and finally found an answer. In short, the issue is that tf-random (as well as Liquid) needs UTF-8 input. And FreeBSD does not use that encoding by default.

The solution

Add appropriate language and encoding tag. to the default: section of your /etc/login.conf

If you your language is en_US then you lang tag would be :lang=en_US.UTF-8:

Before:

default:\
  :passwd_format=md5:\

  *many lines...*

  :umask=022:

After:

default:\
  :passwd_format=md5:\

  *many lines...*

  :umask=022:\
  :lang=en_US.UTF-8:

To reload the changes, run (as root):

cap_mkdb /etc/login.conf

After that relogin to your environment. More details in the mentioned post. Thanks to Patrick Gibson for sharing the solution.

'Welcome to Jekyll'

Tue, 24 Jun 2014 14:32:03 +0000

There is a lot left to be done, but I got bare minimal content migrated to Github Pages from my old blog. I hope to be using this blog mainly to publish documents related to my projects hosted on Github.

Haskell build automation: alternatives to Cabal

Thu, 12 Jun 2014 00:00:00 +0000

In the previous post I've talked about issues that Cabal has. Now it is time to talk about what alternatives there are.

Existing tools: task management

Haskell can work with various task automation tools, starting with good old make. Among the tools written in Haskell nemesis and shake definitely worth mentioning.

One problem with task management tools is that one has to create work flow for them, they don't mandate anything or come with any predefined tasks. A bigger problem is that they still require `global' dependencies, so building with different versions of the same library would be as painful as it is with Cabal.

Some workaround would be to create some project-carving tool, like Jeweler or Hoe in Ruby. That would introduce a work flow and will set up important tasks, like mandatory testing, but will not help with dependencies. Well, actually the tool can incorporate dependency resolution task, but that will make it almost identical to work flow solutions discussed below.

Solutions yet to be implemented: standard work flow

More drastic approach is to implement a work flow based tool similar to Maven. Standard work flows are a great thing, especially when project needs to be packaged in a particular way, or one wants to ensure that tests are ran every time a build is done.

That also has more potential for fixing dependencies, because if fetching dependencies is part of the work flow, then they can be stored and compiled (assuming that they are distributed as source code) locally.

As a solution one can implement a Haskell plugin for an existing tool, like Maven or Gradle. That would be as hard as writing it from scratch, since those are heavily geared towards compiling to Java bytecode. Also, those tools are not implemented in Haskell, and I would like to show that Haskell is can stand up to a project like this.

So I personally really like the idea of implementing a `conventions and standard work flows' build automation tool in Haskell. I will try to give this a shot, but it will be a big effort, so we'll see what happens.

Haskell build automation: Cabal

Mon, 26 May 2014 00:00:00 +0000

Disclaimer

The purpose of this and the following posts is to evaluate options for build automation tools for Haskell, and propose possible solutions for improving developer experience and facilitating test-driven development.

This post is solely based on my opinion, which is affected by my experience with other tools, mostly Maven and Make. I believe that having a good build automation tool makes a big impact on developer experience and code quality. I think improving build automation is easier and more beneficial than enabling continuous integration using a faulty build tool.

I don't have an intention to criticize anyone or the software they are contributing to. I think Cabal is a great tool for creating and sharing packages, but I would like to look at it in relation to build automation.

What is a `build automation tool'

Wikipedia on build automation:

Build automation is the act of scripting or automating a wide variety of tasks that software developers do in their day-to-day activities including things like:

compiling computer source code into binary code
packaging binary code
running tests
deployment to production systems
creating documentation and/or release notes

As for the build automation system, there are various ways to think about that, but what I am looking for is a tool that can run at least build, test and package in one command.

Using Cabal for builds

Packaging and sharing of Haskell projects is traditionally done via Cabal. Cabal is as good as it can get for manipulating Haskell packages and interacting with HackageDB, but it has several shortcomings when it comes to manging builds.

There is an excellent post about why Cabal is not a good package manager. I have to add that it might not pass as a modern build automation tool, either.

Build life cycle

Build life cycle is a set of stages that project passes during build process. It enforces a work flow. For tools that see builds as a set of custom tasks or targets (Make, Rake, Ant) that work flow is defined by the relationships between tasks and there is now default. For tools that are based on conventions and standard life cycles, like Maven, there is a default work flow that can be extended to fit project needs.

Cabal does not fall into either of the categories: it is does not have custom tasks, but does not have a standard work flow either. There is a list of commands in Cabal user guide, (and some of them are obviously included in one another like install and register) but there is no clear definition of what build phases project has to go through before deployment.

Configuration

In addition to .cabal configuration file, there is an extra `configure' step that sometimes needs to be run manually. By default `cabal configure' checks if the dependencies are installed, but it has to be used for some other things, like enabling tests (more on that later):

cabal configure --enable-tests

cabal build

cabal test

I am not sure why that could not have been read from .cabal and automatically included in the build or test phase.

Also, Cabal configuration can not define new build phases, cabal will always stick with the set of command it already has. But there are various situations when having adding a new goal makes a lot of sense than trying to integrate with existing ones.

Haskell Dependencies

There are some issues with how cabalized dependencies are handled in Cabal builds.

Local build dependencies are optional

Unless you are using sandboxes, dependencies that Cabal installs are fairly global (system-wide or user). That can cause version conflicts which you might not be able to resolve.

The solution is to use optional cabal sandbox, that will track installed dependencies for the current copy of the project separately from system-wide libraries. That would help, but will add a few extra manual steps, which will bring us to the next point.

Too much manual intervention

To tell cabal to install dependencies and then build one needs to run two commands:

cabal install --only-dependencies

cabal build

That installs dependencies globally, so if you want to tie them your sandboxed version of the project, you would have to run a few `cabal sandbox' commands on top of that. Sandboxed and non-sandboxed dependency installation is done by the same command, so if you forgot to create a sandbox you are risking messing up your Haskell installation.

Testing

In order to add tests to a cabalized project, one has to provide wrapper code, for example you might have to surround your quickcheck test suite with IO code that reports status via exit codes.

Also, testing is optional, to enable it one has to run a configure command before building the project:

cabal configure --enable-tests

This has to be done even after you have included a reference to test suite in your .cabal configuration file (just mentioning it in the configuration file is not enough). Also, `enable tests' does not stick, if you re-run configure and forget the flag, Cabal will happily disable tests. None of that helps with test-driven development.

TL;DR

Cabal is a great tool for manipulating packages, but it does not enforce a build work flow and requires a significant amount of manual bookkeeping to produce a full build.

(To be potentially continued...)