ridv/gorealis
1
0
Fork 0
Code Issues Pull requests Projects Releases 23 Packages Wiki Activity Actions

Cleaned up the README and added new information to Getting_Started.

Browse source 
Made client point to new location of docker-compose-executor release
This commit is contained in:
Renan DelValle 2016-08-10 11:45:54 -07:00
parent 4e818fdd30
commit 7aa6c07c66
4 changed files with 60 additions and 33 deletions
Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -3,7 +3,7 @@
Go library for communicating with Apache Aurora. Go library for communicating with Apache Aurora.
Named after the northern lights (Aurora Borealis). Named after the northern lights (Aurora Borealis).
### Aurora compatible version ### Aurora version compatibility
Please see [AURORA.VER](./AURORA.VER) to see the latest Aurora version against which this Please see [AURORA.VER](./AURORA.VER) to see the latest Aurora version against which this
library has been tested. Vendoring a working version of this library is highly recommended. library has been tested. Vendoring a working version of this library is highly recommended.

87
docs/Getting_Started.md
View file

@ -1,7 +1,13 @@
# Running custom executors on Aurora # Running custom executors on Aurora
In this instance we will be using the docker-compose executor to demonstrate In this document, we will be using the docker-compose executor to demonstrate
how Aurora can use multiple executors on a single Scheduler. how Aurora can use multiple executors on a single Scheduler. For this guide,
we will be using a vagrant instance to demonstrate the setup process. Many of the same
steps also apply to an Aurora installation made via a package manager. Differences in how to configure
the cluster between the vagrant image and the package manager will be clarified when necessary.
## Spinning up an Aurora instance with Vagrant
Follow the [guide](https://github.com/apache/aurora/blob/master/docs/operations/configuration.md) at the Aurora repository in order to spin up a local cluster.
## Configuring Scheduler to use Docker-Compose executor ## Configuring Scheduler to use Docker-Compose executor
In order use the docker compose executor with Aurora, we must first give the scheduler In order use the docker compose executor with Aurora, we must first give the scheduler
@ -10,20 +16,23 @@ a configuration file that contains information on how to run the executor.
### Configuration file ### Configuration file
The configuration is a JSON file that contains where to find the executor and how to run it. The configuration is a JSON file that contains where to find the executor and how to run it.
More information about how an executor may be configured for consumption by Aurora can be found [here](https://github.com/apache/aurora/blob/master/docs/operations/configuration.md)
under the custom executors section.
A sample config file for the docker-compose executor looks like this: A sample config file for the docker-compose executor looks like this:
``` ```
[ [
{ {
"executor":{ "executor":{
"command":{ "command":{
"value":"java -jar docker-compose-executor_0.0.1.jar", "value":"java -jar docker-compose-executor_0.1.0.jar",
"shell":"true", "shell":"true",
"uris":[ "uris":[
{ {
"cache":false, "cache":false,
"executable":true, "executable":true,
"extract":false, "extract":false,
"value":"https://dl.bintray.com/rdelvalle/mesos-compose-executor/docker-compose-executor_0.0.1.jar" "value":"https://github.com/mesos/docker-compose-executor/releases/download/0.1.0/docker-compose-executor_0.1.0.jar"
} }
] ]
}, },
@ -39,20 +48,32 @@ A sample config file for the docker-compose executor looks like this:
### Configuring Scheduler to run custom executor ### Configuring Scheduler to run custom executor
#### Setting the proper flags `-custom_executor_config` flag and enabling mesos fetcher on jobs #### Setting the proper flags `-custom_executor_config` flag and enabling mesos fetcher on jobs
Some flags need to be set on the Aurora scheduler in order for custom executors to work properly.
The `--custom_executor_config` flag must point to the location of the JSON blob. The `--custom_executor_config` flag must point to the location of the JSON blob.
The `--enable_mesos_fetcher` flag must be set to true in order to allow jobs to fetch resources. The `--enable_mesos_fetcher` flag must be set to true in order to allow jobs to fetch resources.
#### On vagrant #### On vagrant
Modify the file `/etc/init/aurora-scheduler.conf` to include: * Log into the vagrant image by going to the folder at which the Aurora repository
was cloned and running:
```
$ vagrant ssh
```
* Write the sample JSON blob provided above to a file inside the vagrant image.
* Inside the vagrant image, modify the file `/etc/init/aurora-scheduler.conf` to include:
``` ```
--custom_executor_config=<Location of JSON blob> \ --custom_executor_config=<Location of JSON blob> \
--enable_mesos_fetcher=true --enable_mesos_fetcher=true
``` ```
#### On a scheduler installed via package manager #### On a scheduler installed via package manager
Modify the file `/etc/default/aurora-scheduler.conf` to include: * Write the sample JSON blob provided above to a file on the same machine where the scheduler is running.
* Modify the file `/etc/default/aurora-scheduler.conf` to include:
``` ```
AURORA_EXTRA_ARGS="--custom_executor_config=<Location of JSON blob> \ AURORA_EXTRA_ARGS="--custom_executor_config=<Location of JSON blob> \
--enable_mesos_fetcher=true" --enable_mesos_fetcher=true"
@ -72,42 +93,47 @@ the compose executor on Aurora.
#### Ubuntu #### Ubuntu
##### Add PPA and install via apt-get ##### Adding a PPA and install via apt-get
``` ```
$ sudo add-apt-repository ppa:ubuntu-lxc/lxd-stable $ sudo add-apt-repository ppa:ubuntu-lxc/lxd-stable
$ sudo apt-get update $ sudo apt-get update
$ sudo apt-get install golang $ sudo apt-get install golang
``` ```
##### Set GOPATH ##### Setting the GOPATH
Configure the environment to be able to compile and run Go code. Configure the environment to be able to compile and run Go code.
``` ```
$ mkdir $HOME/go $ mkdir $HOME/go
$ echo export GOPATH=$HOME/go >> $HOME/.profile $ echo export GOPATH=$HOME/go >> $HOME/.bashrc
$ echo export PATH=$PATH:$GOPATH/bin >> $HOME/.profile $ echo export PATH=$PATH:$GOPATH/bin >> $HOME/.bashrc
$ echo export PATH=$PATH:$GOROOT/bin >> $HOME/.profile $ echo export PATH=$PATH:$GOROOT/bin >> $HOME/.bashrc
```
Finally we must reload the .bashrc configuration:
```
$ source $HOME/.bashrc
``` ```
### OS X ### OS X
One way to install go on OS X is by using [Homebrew](http://brew.sh/) One way to install go on OS X is by using [Homebrew](http://brew.sh/)
#### Install Homebrew #### Installing Homebrew
Run the following command from the terminal to install Hombrew: Run the following command from the terminal to install Hombrew:
``` ```
$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" $ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
``` ```
#### Install Go using Hombrew #### Installing Go using Hombrew
Run the following command from the terminal to install Go: Run the following command from the terminal to install Go:
``` ```
$ brew install go $ brew install go
``` ```
#### Set the GOPATH #### Setting the GOPATH
Configure the environment to be able to compile and run Go code. Configure the environment to be able to compile and run Go code.
@ -124,9 +150,10 @@ $ echo export PATH=$PATH:$GOROOT/bin >> $HOME/.profile
Download and run the msi installer from https://golang.org/dl/ Download and run the msi installer from https://golang.org/dl/
## Installing Docker Compose ## Installing Docker Compose
To show Aurora's new multi executor feature, we need to use at least one custom executor.
In this case we will be using the [docker-compose-executor](https://github.com/mesos/docker-compose-executor).
In order to run the docker-compose executor, each agent must have docker-compose In order to run the docker-compose executor, each agent must have docker-compose installed on it.
installed on it.
This can be done using pip: This can be done using pip:
``` ```
@ -178,7 +205,7 @@ aurora job create devcluster/www-data/prod/hello hello_world.aurora
Now that we have a thermos job running, it's time to launch a docker-compose job. Now that we have a thermos job running, it's time to launch a docker-compose job.
We'll be using the gorealis library sample client to send a create job request We'll be using the gorealis library sample client to send a create job request
to the scheduler, specifiying that we would like to use the docker-compose executor. to the scheduler, specifying that we would like to use the docker-compose executor.
Furthermore, we will be specifying what resources we need to download in order to Furthermore, we will be specifying what resources we need to download in order to
successfully run a docker compose job. successfully run a docker compose job.
@ -186,19 +213,19 @@ successfully run a docker compose job.
The job configuration in the sample client looks like this: The job configuration in the sample client looks like this:
``` ```
job = realis.NewJob(). job = realis.NewJob().
Environment("prod"). Environment("prod").
Role("vagrant"). Role("vagrant").
Name("docker-compose"). Name("docker-compose").
ExecutorName("docker-compose-executor"). ExecutorName("docker-compose-executor").
ExecutorData("{}"). ExecutorData("{}").
CPU(1). CPU(1).
Ram(64). RAM(64).
Disk(100). Disk(100).
IsService(false). IsService(false).
InstanceCount(1). InstanceCount(1).
AddPorts(1). AddPorts(1).
AddLabel("fileName", "sample-app/sample-app.yml"). AddLabel("fileName", "sample-app/docker-compose.yml").
AddURI("https://dl.bintray.com/rdelvalle/mesos-compose-executor/sample-app.tar.gz", true, true) AddURI("https://dl.bintray.com/rdelvalle/mesos-compose-executor/sample-app.tar.gz", true, true)
``` ```
Now we run the client sending the create job command to Aurora: Now we run the client sending the create job command to Aurora:

2
examples/client.go
View file

@ -79,7 +79,7 @@ func main() {
IsService(false). IsService(false).
InstanceCount(1). InstanceCount(1).
AddPorts(1). AddPorts(1).
AddLabel("fileName", "sample-app/sample-app.yml"). AddLabel("fileName", "sample-app/docker-compose.yml").
AddURI("https://dl.bintray.com/rdelvalle/mesos-compose-executor/sample-app.tar.gz", true, true) AddURI("https://dl.bintray.com/rdelvalle/mesos-compose-executor/sample-app.tar.gz", true, true)
break break
default: default:

2
examples/thermos_payload.json
View file

@ -27,7 +27,7 @@
"ephemeral": false, "ephemeral": false,
"max_failures": 1, "max_failures": 1,
"min_duration": 5, "min_duration": 5,
"cmdline": "\n while true; do\n echo hello world from GoRealis\n sleep 10\n done\n ", "cmdline": "\n while true; do\n echo hello world from gorealis\n sleep 10\n done\n ",
"final": false "final": false
} }
], ],