Mesos#

A short summary of playing around with mesos , marathon and chronos .
Mostly provided by the Mesosphere intro course.

add this option to your Vagrantfile:

      config.vm.box_download_insecure = true

Login with vagrant@localhost:2222 pw=vagrant , or "vagrant ssh"

Install mesos:

    sudo rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm
    sudo yum -y install mesos marathon

Install zookeeper, the distributed configuration service used by mesos:

    sudo rpm -Uvh http://archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm
    sudo yum -y install zookeeper zookeeper-server

Initialize and start Zookeeper:

    sudo -u zookeeper zookeeper-server-initialize --myid=1
    sudo service zookeeper-server start

Install java: yum -y install java-1.8.0-openjdk

Run the interactive zookeeper shell : /usr/lib/zookeeper/bin/zkCli.sh and issue some tests :

Start mesos master and slave :

    systemctl start mesos-master
    systemctl start mesos-slave

    sudo rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm
    sudo yum -y install mesos marathon

Install zookeeper, the distributed configuration service used by mesos:

    sudo rpm -Uvh http://archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm
    sudo yum -y install zookeeper zookeeper-server

Initialize and start Zookeeper:

    sudo -u zookeeper zookeeper-server-initialize --myid=1
    sudo service zookeeper-server start

Install java: yum -y install java-1.8.0-openjdk

Run the interactive zookeeper shell : /usr/lib/zookeeper/bin/zkCli.sh and issue some tests :

Start mesos master and slave :

    systemctl start mesos-master
    systemctl start mesos-slave

Mesos webui available at http://192.168.33.10:5050

Play around a bit with mesos :

    export MASTER=$(mesos-resolve `cat /etc/mesos/zk` 2>/dev/null)
    mesos help
    

Bring up a second node, node2 at 192.168.33.12 :

Install mesos:

    sudo rpm -Uvh http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm
    sudo yum -y install mesos marathon

Install zookeeper, the distributed configuration service used by mesos:

    sudo rpm -Uvh http://archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm
    sudo yum -y install zookeeper zookeeper-server

Initialize and start Zookeeper:

    sudo -u zookeeper zookeeper-server-initialize --myid=1
    sudo service zookeeper-server start

Run the interactive zookeeper shell : /usr/lib/zookeeper/bin/zkCli.sh and issue some tests :

Edit zookeeper config at /etc/mesos/zk, change the IP address to the address of the master.

Start mesos slave :

    sudo systemctl start mesos-slave

Make sure the nodes are DNS accessible (update /etc/hosts) . Logging of marathon, be default, goes to syslog (/var/log/messages)

Running Tasks always have a port, and this port is webaccessible giving you access to stdout and stderr.

Messing with the marathon REST api (see Marathon REST api

• http://192.168.33.10:8080/v2/apps/test ==> more detail on app "test"

Delete an app: curl -X DELETE http://192.168.33.10:8080/v2/apps/test | python -m json.tool

Create an app by posting the following data in (file app1.json) :

    {
  "id": "/app1",
  "cmd": "python -m SimpleHTTPServer $PORT",
  "args": null,
  "user": null,
  "env": {},
  "instances": 3,
  "cpus": 0.9,
  "mem": 16.0,
  "disk": 10.0,
  "executor": "",
  "constraints": [],
  "uris": ["/testapp"],
  "storeUrls": [],
  "ports": [10000],
  "requirePorts": false,
  "backoffSeconds": 1,
  "backoffFactor": 1.15,
  "maxLaunchDelaySeconds": 3600,
  "container": null,
  "healthChecks": [],
  "dependencies": [],
  "upgradeStrategy": {
    "minimumHealthCapacity": 1.0,
    "maximumOverCapacity": 1.0
  }
}

curl -v -H "Content-Type: application/json" -X POST --data @app1.json http://192.168.33.10:8080/v2/apps

Now install chronos (the cron for mesos) :

    sudo yum -y install chronos
    sudo service chronos start

Chronos installs as a mesos framework, like marathon does. (marathon is a sort of init.d for mesos) Chronos is available at http://192.168.33.10:4400/

Install the mesos command line utility :

    curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
    sudo python get-pip.py
    sudo pip install virtualenv
    sudo pip install mesos.cli

Logging#

Create /etc/rsyslog.d/mesos.conf with following content :

if $programname == 'marathon' then {
   action(type="omfile" file="/var/log/mesos/marathon.log")
}

if $programname == 'chronos' then {
   action(type="omfile" file="/var/log/mesos/chronos.log")
}

if $programname == 'mesos-master' then {
   action(type="omfile" file="/var/log/mesos/mesos-master.log")
}

if $programname == 'mesos-slave' then {
   action(type="omfile" file="/var/log/mesos/mesos-slave.log")
}

And look at /var/log/mesos/ for the resulting files.

Authentication#

See the Mesos configuration documentation at :

• https://docs.mesosphere.com/reference/mesos-master/
• https://docs.mesosphere.com/reference/mesos-slave/
• http://mesos.apache.org/documentation/latest/configuration/

I got it working by creating the following files (and restarting master and slave), as the doc says you can create files for flags :

A file named the same name as the flag may be placed in the /etc/mesos-master directory. So a /etc/mesos-master/hostname file containing the value of 10.141.141.10 is like running the master with the option --hostname=10.141.141.10 :

/etc/mesos/mesos-master/authenticate  ==> true
/etc/mesos/mesos-master/authenticate_slaves  ==> true
/etc/mesos/mesos-master/credentials  ==> /etc/mesos/mesos-config/mesos-master.passwd
/etc/mesos/mesos-slave/credential ==> /etc/mesos/mesos-config/mesos-slave.passwd
/etc/mesos/mesos-config/mesos-master.passwd  ==> user password
/etc/mesos/mesos-config/mesos-slave.passwd  ==> user password