The NASA Space Apps Challenge

Adam and I were virtual participants in NASA’s Space Apps Challenge, a weekend global hackathon the weekend of April 21st encouraging citizen scientists to “solve current challenges relevant to both space exploration and social need”. The event had teams from all 7 continents, including researchers from McMurdo Station, Antarctica, as well as the the International Space Station.

Space Apps Challenge map

Space Apps Challenge Hackathon

NASA seeded candidate projects prior to the event in 4 categories: software, open hardware, citizen science, and data visualization. Teams could also submit their own proposals. You can view the full list of candidate projects.

I was pleased to see the NASA Planetary Data System (PDS) catalog listed as a seed project. I had looked into programmatically downloading the images available through PDS 6 months ago and discovered that they are stored server-side in the VICAR format, which was unsupported by any open source image processing utilities I could find (ImageMagick nominally has support, but produces garbage images on conversion to other formats). This hackathon was a perfect opportunity to write a utility to convert VICAR into a more accessible format.

vicar2png

Our submission for this hackathon was vicar2png, a utility written in Python with the pypng library to convert VICAR files to the popular PNG image format.

VICAR (the Video Image Communication And Retrieval format), has been used since 1966 for NASA mission images. Check out N1349602309_2.IMG from the Cassini-Huygens mission for an example of a VICAR file. NASA maintains a format specification that we used to implement our solution.

VICAR file example

VICAR file from the Cassini-Huygens mission

You can view all of the hackathon submissions.

vicar2png makes it easy for anyone to view, enjoy, and remix NASA’s mission image data easily by converting VICAR files to PNG images. We also hope that making this simple, open source Python implementation available makes it easier for general-purpose open source image processing tools to add VICAR support.

Hackathon deliverables

vicar2png was one of 37 entries nominated for global judging, out of 111 hackathon submissions!

Saturn's moon Phoebe, converted from VICAR data from the Cassini-Huygens mission

Saturn's moon Phoebe, converted from VICAR data from the Cassini-Huygens mission

Space Apps Challenge Global Judging

For global judging we had to create a short video explaining and demoing our submission.

Global competition deliverables

Global judging was streamed on uStream; you can view the archived video here. They announced 5 winners:

  1. Most Inspiring: Planet Hopper
    Visualize exoplanets using Kepler data.
  2. Best Use of Data: vicar2png
    View, enjoy, and remix NASA’s mission image data easily by converting VICAR files to the popular PNG image format.
  3. Most Disruptive: Growing Fruits: Pineapple Project
    Determine the optimal crop for your community based on rainfall, latitude, elevation, and pH.
  4. Most Innovative: Strange Desk
    Share strange events in your community with other citizen scientists.
  5. Galactic Impact: Growers Nation
    Use location, climate, and growing data to find appropriate crops for unused land.
  6. “People’s Choice”: Bit Harvester
    Control energy systems in remote areas via SMS.

Best use of Data — awesome!

Winners got a miniature spaceship and an interview with Gov 2.0.

We were satisfied to have scoped a hackathon project to meet an actual need we had and could finish in a weekend. Thank you Space Apps Challenge organizers for giving us this opportunity!

Saturn's Rings, converted from VICAR data from the Cassini-Huygens missionSaturn's moon Phoebe, converted from VICAR data from the Cassini-Huygens missionJupiter, converted from VICAR data from the Cassini-Huygens mission

Press for global winners

Future work

We’d like to use vicar2png as a reference implementation for fixing VICAR support in ImageMagick and adding VICAR support to other open source image processing utilities.

Additionally, NASA uses several other obscure image formats for some of its mission data. In particular, a lot of the Mars Rover images are in the “PDS” image format. As with VICAR, we’d like to write a PDS converter as a step towards increasing the accessibility of the Mars Rover data and to facilitate adding PDS support to other image processing tools.

LD_TRACE_LOADED_OBJECTS, but not really

Zephyr was talking about the ability to dispute CVE assignments, and CVE-2009-5064: “glibc: ldd unexpected code execution issue” came up as an example. MITRE says:

** DISPUTED ** ldd in the GNU C Library (aka glibc or libc6) 2.13 and earlier allows local users to gain privileges via a Trojan horse executable file linked with a modified loader that omits certain LD_TRACE_LOADED_OBJECTS checks. NOTE: the GNU C Library vendor states “This is just nonsense. There are a gazillion other ways to introduce code if people are downloading arbitrary binaries and install them in appropriate directories or set LD_LIBRARY_PATH etc.”

This got me curious about the LD_TRACE_LOADED_OBJECTS environment variable and what the libc authors were disputing. Googling around got me to this 2009 article by Peteris Krumins. The premise is:

  1. ldd is mostly just a wrapper around invoking the dynamic linker on an executable with LD_TRACE_LOADED_OBJECTS set. The GNU dynamic linker (e.g. /lib/ld-linux.so.2 or /lib64/ld-linux-x86-64.so.2) checks for LD_TRACE_LOADED_OBJECTS and if set prints the shared libraries required by the executable instead of running it.
  2. If you can get a sysadmin to run ldd on a malicious executable which was compiled to use a dynamic linker that doesn’t check for LD_TRACE_LOADED_OBJECTS, that malicious executable will run instead of just having its shared library dependencies printed to stdout.

The blog post jumps through quite a few hoops to use a modified uClibc dynamic linker that doesn’t try to check for LD_TRACE_LOADED_OBJECTS. Comments on the post pointed out that there was an easier way to do this:

1. Create a test executable

$ cat > test_ld.c
#include <stdio.h>

int main() {
  if (getenv("LD_TRACE_LOADED_OBJECTS")) {
    printf("LD_TRACE_LOADED_OBJECTS is set, and yet I am executing!\n");
  }
  else {
    printf("LD_TRACE_LOADED_OBJECTS is not set, so I am executing.\n");
  }
  return 0;
}

2. Statically compile the test executable into what will be our dynamic linker

$ gcc -static -o test_ld.so test_ld.c

3. Tell the linker to specify our test executable as the dynamic linker to use for our test executable

$ gcc -Wl,-dynamic-linker,test_ld.so test_ld.c -o test_ld

The -Wl,<option>,<option args> syntax passes <option> and <option args> on to the linker.

The dynamic linker is written into the PT_INTERP program header in the resulting ELF executable. Chapter 2 of the ELF format specification describes dynamic linking in detail.

4. Run our executable which specifies a custom dynamic linker

$ ./test_ld
LD_TRACE_LOADED_OBJECTS is not set, so I am executing.
$ LD_TRACE_LOADED_OBJECTS=1 ./test_ld
LD_TRACE_LOADED_OBJECTS is set, and yet I am executing!

And we see that running test_ld, even when LD_TRACE_LOADED_OBJECTS is set, the program executes. What is actually going on here is that test_ld.so is getting invoked as the dynamic linker. test_ld never actually executes, since test_ld.so didn’t set up the execution environment and jump to the _start of the executable, which is what a dynamic linker is supposed to do.

I don’t like this example, though, because using the test executable as both the dynamic linker and target executable makes it less obvious what is actually executing. It also over-emphasizes the security implications of LD_TRACE_LOADED_OBJECTS: if you are running or probing untrusted executables, you don’t need LD_TRACE_LOADED_OBJECTS to get yourself into trouble.

This is a clearer example:

1. Create and compile a stub dynamic linker

$ cat > stub_ld.c 
int main() {
  return 27;
}
$ gcc -static -o stub_ld.so stub_ld.c

2. Create and compile a test executable that specifies our stub linker as the dynamic linker

$ cat test.c 
#include <stdio.h>

int main()
{
  printf("Hello world\n");
  return 0;
} 
$ gcc -Wl,-dynamic-linker,stub_ld.so test.c -o test

3. Run the test executable

$ ./test
$ echo $?
27

The fact that "Hello world" isn’t printed makes it clear that only stub_ld.so executes.

ldd security improvements

The specific ldd trick from Krumins’ 2009 article — social engineering a sysadmin into running ldd on a executable that invokes a custom dynamic linker that ignores LD_TRACE_LOADED_OBJECTS — doesn’t work on newer systems. On older systems (e.g. RHEL 4), ldd will use the dynamic linker specified in the executable. On newer systems (e.g. Ubuntu Oneiric) ldd hardcodes and will only use the GNU dynamic linkers.

Here are the relevant snippets from old and new ldd versions for comparison, with extraneous code elided.

Older ldd

RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2"

try_trace() {
  eval $add_env '"$@"' | cat
}

for rtld in ${RTLDLIST}; do                                                                                                                                                                                      
  if test -x $rtld; then                                                                                                                                                                                         
    verify_out=`${rtld} --verify "$file"`
    ret=$?
    case $ret in
    [02]) RTLD=${rtld}; break;;                                                                                                                                                                                  
    esac
  fi
done
case $ret in
0)
  try_trace "$file"

Note that if verification returns 0, try_trace just evals the executable directly, so the dynamic linker specified in the executable is invoked. With this ldd we get:

$ ldd ./test
$ echo $?
27

Newer ldd

RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2"

try_trace() {
  eval $add_env '"$@"' | cat
}

for rtld in ${RTLDLIST}; do                                                                                                                                                                                      
  if test -x $rtld; then                                                                                                                                                                                         
    verify_out=`${rtld} --verify "$file"`
    ret=$?
    case $ret in
    [02]) RTLD=${rtld}; break;;                                                                                                                                                                                  
    esac
  fi
done
case $ret in
0|2)
  try_trace "$RTLD" "$file" || result=1

In this case try_trace always use the GNU dynamic linker for our architecture to run the executable, even if something else was specified in PT_INTERP. With this ldd we get:

$ ldd ./test
	linux-vdso.so.1 =>  (0x00007fff4cbff000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd51eec9000)
	stub_ld.so => /lib64/ld-linux-x86-64.so.2 (0x00007fd51f287000)

which is the same as if we hadn't specified a custom dynamic linker. Note that our stub_ld.so is printed as living at the location of the GNU dynamic linker /lib64/ld-linux-x86-64.so.2.

I'm not sure where the newer version of ldd is coming from, though. dpkg -S `which ldd` says libc-bin, which comes from eglibc, is the package supplying ldd. I checked out the eglibc and glibc source repositories and neither have the updated ldd in their histories.

The Indianapolis Python Workshop and Indiana LinuxFest

Attendees practicing at the Indianapolis Python Workshop

I flew out to Indianapolis for the weekend of April 13th to help run the first Indianapolis Python Workshop, which was co-located with Indiana LinuxFest.

This was the second city to use our Boston Python Workshop grant from the Python Software Foundation to help bootstrap Python workshops for women with new user groups in the US. Catherine Devlin and Mel Chua were the locals and main organizers leading the show.

Running the event as part of Indiana LinuxFest presented some challenges, but the attendees were great, including some very sweet father-daughter teams. This part of the country has some strong regional events, including PyOhio and Ohio LinuxFest, and I look forward to seeing more great outreach initiatives from Catherine and Mel that capitalize on this fact.

Catherine has a write-up on the event on her blog, and I took a few photos.

Saturday afternoon instructions at the Indianapolis Python Workshop

Since I was already going to be at Indiana LinuxFest for the workshop, I got the chance to give a rehash of my “The Internet Shouldn’t Work, Networking 101″ talk. It is a beginner-level tour through the Internet’s history, governance, protocols, and current events, with telnet, traceroute, and wireshark demos interspersed. Here are my slides from the talk.

Sunday Morning Linux Review, a “weekly podcast in a news format that focuses on Linux and Open source topics”, asked if I could give a short interview for the show while I was in town. Interviewer Tony Bemus and I chatted a bit about the workshop, school, Ksplice, and Linux, starting around 41:15 in episode 27 of the show, my first ever appearance on a podcast.