An (not so big) intro to DOCKER

Docker is a platform to develop, ship, and run applications. According to the Wikipedia, Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation of operating system–level virtualization on Linux. Docker mainly facilitates to run almost any application inside a securely isolated container in multi-tenant (or simply, multi-container) environments. Since there is no hypervisor like the traditional approach, this lightweight containers can really get better performance.

Docker uses resource isolation features of the Linux kernel such as cgroups and kernel namespaces to allow independent “containers” to run within a single Linux instance, avoiding the overhead of starting virtual machines. Linux kernel’s namespaces completely isolate an application’s view of the operating environment, including process trees, network, user IDs and mounted file systems, while cgroups provide resource isolation, including the CPU, memory, block I/O and network. Docker includes the libcontainer library as a reference implementation for containers, and builds on top of libvirt, LXC (Linux containers) and systemdnspawn, which provide interfaces to the facilities provided by the Linux kernel.

With just enough of the definition, lets get our hand dirty with a first run in the docker.
You can just go to the Try DOCKER and run the below commands:

Docker prompt and Version:

you@tutorial:~$ docker
Usage: Docker [OPTIONS] COMMAND [arg...]
-H="127.0.0.1:4243": Host:port to bind/connect to

A self-sufficient runtime for linux containers.

Commands:

attach    Attach to a running container
build     Build a container from a Dockerfile
commit    Create a new image from a container's changes
diff      Inspect changes on a container's filesystem
export    Stream the contents of a container as a tar archive
history   Show the history of an image
images    List images
import    Create a new filesystem image from the contents of a tarball
info      Display system-wide information
insert    Insert a file in an image
inspect   Return low-level information on a container
kill      Kill a running container
login     Register or Login to the Docker registry server
logs      Fetch the logs of a container
port      Lookup the public-facing port which is NAT-ed to PRIVATE_PORT
ps        List containers
pull      Pull an image or a repository from the Docker registry server
push      Push an image or a repository to the Docker registry server
restart   Restart a running container
rm        Remove a container
rmi       Remove an image
run       Run a command in a new container
search    Search for an image in the Docker index
start     Start a stopped container
stop      Stop a running container
tag       Tag an image into a repository
version   Show the Docker version information
wait      Block until a container stops, then print its exit code

Pulling from Repository:

Lets search for a image called tutorial in the repository and pull it into our environment.

you@tutorial:~$ docker search tutorial
Found 1 results matching your query ("tutorial")
NAME                      DESCRIPTION
learn/tutorial            An image for the interactive tutorial
you@tutorial:~$ docker pull learn/tutorial
Pulling repository learn/tutorial from https://index.docker.io/v1
Pulling image 8dbd9e392a964056420e5d58ca5cc376ef18e2de93b5cc90e868a1bbc8318c1c (precise) from ubuntu
Pulling image b750fe79269d2ec9a3c593ef05b4332b1d1a02a62b4accb2c21d589ff2f5f2dc (12.10) from ubuntu
Pulling image 27cf784147099545 () from tutorial

Running the Helloworld:

Lets run the very first helloworld thing with echo command:

you@tutorial:~$ docker run learn/tutorial echo "My Worderful HelloWorld in Docker"
My Worderful HelloWorld in Docker

Install package and commit:

Lets install the ping tools from apt-get repo. Once that is done, lets save this image with a new name "ping"
Once the commit is done, we can push it back to the [docker-hub] (https://hub.docker.com)

you@tutorial:~$ docker run learn/tutorial apt-get install -y ping
Reading package lists...
Building dependency tree...
The following NEW packages will be installed:
  iputils-ping
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 56.1 kB of archives.
After this operation, 143 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ precise/main iputils-ping amd64 3:20101006-1ubuntu1 [56.1 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 56.1 kB in 1s (50.3 kB/s)
Selecting previously unselected package iputils-ping.
(Reading database ... 7545 files and directories currently installed.)
Unpacking iputils-ping (from .../iputils-ping_3%3a20101006-1ubuntu1_amd64.deb) ...
Setting up iputils-ping (3:20101006-1ubuntu1) ...
you@tutorial:~$ docker run learn/ping ping www.google.com
PING www.google.com (74.125.239.129) 56(84) bytes of data.
64 bytes from nuq05s02-in-f20.1e100.net (74.125.239.148): icmp_req=1 ttl=55 time=2.23 ms
64 bytes from nuq05s02-in-f20.1e100.net (74.125.239.148): icmp_req=2 ttl=55 time=2.30 ms
64 bytes from nuq05s02-in-f20.1e100.net (74.125.239.148): icmp_req=3 ttl=55 time=2.27 ms
64 bytes from nuq05s02-in-f20.1e100.net (74.125.239.148): icmp_req=4 ttl=55 time=2.30 ms

you@tutorial:~$ docker ps -l
ID                  IMAGE               COMMAND                CREATED             STATUS              PORTS
6982a9948422        ubuntu:12.04        apt-get install ping   1 minute ago        Exit 0

you@tutorial:~$ docker commit 698 learn/ping
effb66b31edb

you@tutorial:~$ docker push learn/ping

Detail about a container:

We can easily inspect the details of a running container by using the inspect command. We don’t need to write the whole ID, just first 3~4 characters will suffice.

you@tutorial:~$ docker ps
ID                  IMAGE               COMMAND               CREATED             STATUS              PORTS
efefdc74a1d5        learn/ping:latest   ping www.google.com   37 seconds ago      Up 36 seconds
you@tutorial:~$ docker inspect efe
[2013/07/30 01:52:26 GET /v1.3/containers/efef/json
{
  "ID": "efefdc74a1d5900d7d7a74740e5261c09f5f42b6dae58ded6a1fde1cde7f4ac5",
  "Created": "2013-07-30T00:54:12.417119736Z",
  "Path": "ping",
  "Args": [
      "www.google.com"
  ],
  "Config": {
      "Hostname": "efefdc74a1d5",
      "User": "",
      "Memory": 0,
      "MemorySwap": 0,
      "CpuShares": 0,
      "AttachStdin": false,
      "AttachStdout": true,
      "AttachStderr": true,
      "PortSpecs": null,
      "Tty": false,
      "OpenStdin": false,
      "StdinOnce": false,
      "Env": null,
      "Cmd": [
          "ping",
          "www.google.com"
      ],
      "Dns": null,
      "Image": "learn/ping",
      "Volumes": null,
      "VolumesFrom": "",
      "Entrypoint": null
  },
  "State": {
      "Running": true,
      "Pid": 22249,
      "ExitCode": 0,
      "StartedAt": "2013-07-30T00:54:12.424817715Z",
      "Ghost": false
  },
  "Image": "a1dbb48ce764c6651f5af98b46ed052a5f751233d731b645a6c57f91a4cb7158",
  "NetworkSettings": {
      "IPAddress": "172.16.42.6",
      "IPPrefixLen": 24,
      "Gateway": "172.16.42.1",
      "Bridge": "docker0",
      "PortMapping": {
          "Tcp": {},
          "Udp": {}
      }
  },
  "SysInitPath": "/usr/bin/docker",
  "ResolvConfPath": "/etc/resolv.conf",
  "Volumes": {},
  "VolumesRW": {}

Reference:
1. [Docker Documentation] (https://docs.docker.com/)
2. [Understanding Docker] (https://docs.docker.com/introduction/understanding-docker/)

pdb – The Python Debugger

Debugging with pdb

Very first thing, read this blog by Steve Ferg’s which explains pdb in details.

Here are the summary of the necessary commands:

import pdb
setting up break points by pdb.set_trace()

in-debug keys:

- n (next)
- ENTER (repeat previous)
- q (quit)
- p <variable> (print value)
- c (continue) 
- l (list where you are)
- s (step into subroutine)
- r (continue till the end of the subroutine)
- ! <python command>

Bulk Renaming Python Script

During a map reduce job, i was in need of bulk renaming input files to a specific pattern. Hence, wrote the below simple python script that can just do the job.

import os
import sys

run_ok = False
if len(sys.argv) == 5:
    directory_name = sys.argv[1]
    old_extension = sys.argv[2]
    name_pattern  = sys.argv[3]
    new_extension = sys.argv[4]
    print "Running jobs on directory: {} having extensions {}".format(directory_name, old_extension)
    run_ok = True

else:
    print "SYNTAX : python file_rename.py directory_name matching_extension new_name_pattern new_extension"
    pass

if (run_ok):
    list = os.listdir( directory_name )
    counter = 1
    for old_file in list:
        if old_extension in old_file:
            new_file = name_pattern + str( counter ) + "." + new_extension
            os.rename( directory_name + "/"+ old_file, directory_name + "/" + new_file )
            print "file renamed from {} to {}".format( old_file, new_file )
            counter += 1

stopping swift services using python script

import sys
import shutil
import os
import subprocess, signal 

'''
    format : python stop_service {proxy|object}
    List the process and find server pid to kill
'''
service = 'swift-'+sys.argv[1]+'-server'
print service
proc_list = subprocess.Popen(['ps','-ef'],stdout = subprocess.PIPE)
out,err = proc_list.communicate()
for line in out.splitlines():
    #print line
    if (service in line):
        pid = int (line.split(None,2)[1])
	os.kill(pid,signal.SIGKILL)

print 'please run the {} service  manually'.format(service)