Compare commits

...

56 commits
v0.1.0 ... main

Author SHA1 Message Date
lawwong1
464db4e5cc
Implement Australis api to restart a task (#43) 2022-09-22 20:05:19 -07:00
ananaysingh
589e337e28
Updated kill.go (#42)
in killTask function, null check removed for instances, as it is a mandatory parameter in the killTask command. Also made changes to improve language consistency
2022-09-15 10:37:02 -07:00
lawwong1
7b298f7a35
Generate Docs for Australis (#41)
Co-authored-by: Lawrence Wong <lawwong@paypal.com>
2022-09-09 09:07:39 -07:00
nhatle
8ed6f5a773 release 1.0.5 2022-08-31 15:05:08 -07:00
nhatle
b08640f26a fix duplicated flags -i 2022-08-31 15:04:55 -07:00
nhatle
3d49194ccd release 1.0.5 2022-08-31 13:44:08 -07:00
lawwong1
a19c7e1fb5
API to get master nodes for mesos and aurora (#38) 2022-08-24 10:25:49 -07:00
ananaysingh
b0743636a1
KillTask API (#37) 2022-08-18 10:27:57 -07:00
lawwong1
2c703978bb
given a task status, list all tasks that have that status in the cluster (#35) 2022-08-15 13:35:04 -07:00
nhatle
0a3357e571 relase v1.0.4 2022-08-02 16:21:11 -07:00
nhatle
47644c13c2 release 1.0.4 2022-08-02 10:48:45 -07:00
Tan N. Le
14691698f6
fetch capacity and simulate task fitting (#33) 2022-08-02 10:37:31 -07:00
lenhattan86
66bd6308ce update version 1.0.3 2021-10-20 14:46:53 -07:00
Tan N. Le
48db32c858
update CI to compile on PRs (#28) 2021-10-20 14:12:30 -07:00
Tan N. Le
0d5651028c
update gorealis version (#27) 2021-10-20 14:00:15 -07:00
lenhattan86
7bcf6d3f68
Merge pull request #25 from lenhattan86/add_more_in_TaskConfig
Add tier and production in task config
2021-10-06 19:21:21 -07:00
lenhattan86
c0c79997d0 add tier and production 2021-10-05 20:43:39 -07:00
lenhattan86
c6adde03af
Merge branch 'aurora-scheduler:main' into master 2021-10-05 17:18:47 -07:00
lenhattan86
23e9c904d6
Adds fetch quota command (#24) 2021-09-16 18:36:42 -07:00
lenhattan86
8ae23aad30
Adds priority into job config (#23) 2021-09-16 18:36:13 -07:00
lenhattan86
aa7e052d7a
Merge branch 'aurora-scheduler:main' into master 2021-09-07 12:31:05 -07:00
Renán Del Valle
13ae459d2a
Bumps up current version. 2021-01-11 17:59:38 -08:00
Renán Del Valle
154a4e1e87
v1.0.2 Release 2021-01-11 17:59:13 -08:00
Renán Del Valle
30a5136b24
Bumping up backported Thrift library. 2021-01-11 17:53:01 -08:00
lenhattan86
438b5360c1
Merge pull request #2 from aurora-scheduler/main
fetch mesos leader.
2020-12-09 15:04:10 -08:00
Abdul Qadeer
b21211552f
Default to local Mesos agent if no ZK nodes are provided (#20)
* Update go version

* Fetches master flag from local mesos agent in order to get current leader from ZK if no arguments are provided.
2020-12-01 09:49:03 -08:00
lenhattan86
2f05111192
Fetch mesos leader through zookeeper (#19)
Adds support for fetching current Mesos leader through Zookeeper
2020-11-17 12:39:51 -08:00
lenhattan86
8ebf66426a
Merge pull request #1 from aurora-scheduler/main
pull from the upstream
2020-11-13 10:48:05 -08:00
Renán I. Del Valle
be6c458f23
Documentation update (#17)
* Updates documentation automatically generated by Cobra. Deletes some old, outdated documentation.
2020-11-05 15:46:07 -08:00
Renán I. Del Valle
9fa6edaa3e
Adds support for creating jobs with limit constraints and value constraints. (#16)
* Adds support for creating jobs with limit constraints and value constraints.
2020-11-04 11:46:53 -08:00
Renan DelValle
464ef72e6b
Changing style of setting up github actions env due to the deprecation of the old way. 2020-10-14 20:53:32 -07:00
Renan DelValle
cea39ddd88
Fixing all files to conform with goimports. 2020-10-14 20:46:50 -07:00
Renan DelValle
99af97736c
Fixing goimports call 2020-10-14 20:46:32 -07:00
Renan DelValle
7e7c887231
Bumping up go setup in CI to v2. 2020-10-14 20:32:34 -07:00
Renan DelValle
a7750c5c98
Adding simple goimports check and build check CI. 2020-10-14 20:27:25 -07:00
Renan DelValle
7967188513
Fixing issue where duration variable was accidentally multiplied by seconds again. 2020-10-14 20:17:43 -07:00
lenhattan86
bb370151e6
Add gorealis timeout flag (#14)
Flag was added to control gorealis timeout
2020-10-08 09:01:56 -07:00
Renan DelValle
2a76e3e2b4
1.0.1 release 2020-09-30 16:47:53 -07:00
Renan DelValle
d5d19cf448
Changing default value of monitor to true. 2020-09-30 16:46:14 -07:00
Renan DelValle
deaddd2953
Bumping up release number in root.go so that version command returns the right version. Fixing some bugs in the debian packaging scripts. Bumping up golang version we're building with to 1.15.2 2020-09-30 16:34:02 -07:00
lenhattan86
5efbbce6c7
skip monitoring create/kill jobs (#13)
Added flag -m and --monitor that can be set in order to monitor a job creation or a job kill. By default monitor is set to true.
2020-09-30 16:32:53 -07:00
Renan DelValle
82fe22e013
Adding minimum instances to test data for update. Changing YAML de-serializing field name to match the name od the struct member and to decrease ambiguity. 2020-05-07 20:19:26 -07:00
Renan DelValle
090fdf1715
Removing update settings from cron example as they're not needed. 2020-05-07 16:46:05 -07:00
Renan DelValle
13fbbce816
Adding rudimentary test for unmarshaling cron job. 2020-05-07 16:37:19 -07:00
Renan DelValle
d06530d0ca
Adding sample update job. 2020-05-07 16:31:32 -07:00
Renan DelValle
2df6672acc
Adding schedule command to schedule cron jobs.
Adding example of a cron job.

Removing functions since they've been moved elsewhere.
2020-05-07 16:28:55 -07:00
Renan DelValle
620586fa73
Upgrading documentation automatically generated by cobra. 2020-05-07 15:44:27 -07:00
Renán I. Del Valle
4806936c71
Adding ability to start an update (#12)
* Adding ability to start an update.

* Refactoring Job parsing code to be re-usable.
2020-05-07 15:40:22 -07:00
Renan DelValle
d7db155d88
Refactoring project to be more similar to other go project. 2020-05-07 10:09:45 -07:00
Renan DelValle
bbccabcd27
Moving internal types to the right location. 2020-05-06 18:30:22 -07:00
Renán I. Del Valle
e3e7bc1110
Upgrading dependencies to the latest available as well as bumping up go version to 1.14. (#11) 2020-05-05 23:17:40 -07:00
Renán I. Del Valle
b068369e86
Removing vendored files as they are now cached by go proxy. (#10)
We no longer need vendoring as packages are cached in case they go missing or move from their current location.
2020-05-05 23:09:37 -07:00
Renan DelValle
ca2fab72fd
Increasing version to 0.1.2 unreleased. 2020-02-19 12:40:05 -08:00
Renan DelValle
6603e24ec8
Releasing version 0.1.1. 2020-02-19 12:36:11 -08:00
Renán I. Del Valle
5f510b74ea
Changing dependencies to point to aurora-scheduler organization (#9)
* Removing support for dep as we should now only be building with go1.13+ and mod.

* Ignoring go.sum being created. Use the database instead.

* Upgrading dependencies to their latest versions.

* Replacing thrift dependency temporarily with a patched thrift dependency while we wait for 0.14.0 to be released.

* Change location of gorealis to point to auora-scheduler repository.

* Update .gitignore to contain DS_Store.

* Adding changelog.

* Putting unreleased back into the changelog as this isn't a release PR.
2020-02-19 12:18:50 -08:00
Renan DelValle
08c68935e7
Changing version to unreleased 0.1.1 2020-02-10 12:05:32 -08:00
4376 changed files with 2429 additions and 1438261 deletions

29
.github/workflows/ci.yml vendored Normal file
View file

@ -0,0 +1,29 @@
name: CI
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Setup Go for use with actions
uses: actions/setup-go@v2
with:
go-version: 1.15
- name: Install goimports
run: go get golang.org/x/tools/cmd/goimports
- name: Set env with list of directories in repo containin go code
run: echo GO_USR_DIRS=$(go list -f {{.Dir}} ./... | grep -E -v "/gen-go/|/vendor/") >> $GITHUB_ENV
- name: Run goimports check
run: test -z `for d in $GO_USR_DIRS; do goimports -d "$d" | tee /dev/stderr; done`
- name: Run tests
run: go build -o australis *.go

5
.gitignore vendored
View file

@ -6,3 +6,8 @@ debian/australis.substvars
build/
dist/
.idea/
# Use checksum database
go.sum
.DS_Store

View file

@ -1,3 +1,40 @@
1.0.6 (unreleased)
1.0.5
* fetch mesos & aurora master nodes
* kill an instance from a job
1.0.4
* fetch free capacity
* simulate task fitting - compute how many tasks can be fit in the remaining capacity
1.0.3
* update CI to compile on PRs
* Add tier and production in task config
* Add fetch quota command
* Add priority into job config
1.0.2
* Fixing broken Thrift dependency by bumping up backported version to thrift v0.13.2
1.0.1
* Added flag -m and --monitor that can be set in order to monitor a job creation or a job kill. By default monitor is set to true.
1.0.0
* First stable release.
0.1.1
* Removed support for building with dep
* Upgraded depdencies to their latest versions.
* gorealis v2 now lives in the aurora-scheduler organization and dependencies have been updated to reflect that.
0.1.0
* Adding support for drain sub-command to take in JSON list from stdin or from a specified file.

218
Gopkg.lock generated
View file

@ -1,218 +0,0 @@
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
branch = "0.12.0"
digest = "1:89696c38cec777120b8b1bb5e2d363d655cf2e1e7d8c851919aaa0fd576d9b86"
name = "git.apache.org/thrift.git"
packages = ["lib/go/thrift"]
pruneopts = ""
revision = "384647d290e2e4a55a14b1b7ef1b7e66293a2c33"
[[projects]]
digest = "1:eb53021a8aa3f599d29c7102e65026242bdedce998a54837dc67f14b6a97c5fd"
name = "github.com/fsnotify/fsnotify"
packages = ["."]
pruneopts = ""
revision = "c2828203cd70a50dcccfb2761f8b1f8ceef9a8e9"
version = "v1.4.7"
[[projects]]
digest = "1:d14365c51dd1d34d5c79833ec91413bfbb166be978724f15701e17080dc06dec"
name = "github.com/hashicorp/hcl"
packages = [
".",
"hcl/ast",
"hcl/parser",
"hcl/printer",
"hcl/scanner",
"hcl/strconv",
"hcl/token",
"json/parser",
"json/scanner",
"json/token",
]
pruneopts = ""
revision = "8cb6e5b959231cc1119e43259c4a608f9c51a241"
version = "v1.0.0"
[[projects]]
digest = "1:870d441fe217b8e689d7949fef6e43efbc787e50f200cb1e70dbca9204a1d6be"
name = "github.com/inconshreveable/mousetrap"
packages = ["."]
pruneopts = ""
revision = "76626ae9c91c4f2a10f34cad8ce83ea42c93bb75"
version = "v1.0"
[[projects]]
digest = "1:6a874e3ddfb9db2b42bd8c85b6875407c702fa868eed20634ff489bc896ccfd3"
name = "github.com/konsorten/go-windows-terminal-sequences"
packages = ["."]
pruneopts = ""
revision = "5c8c8bd35d3832f5d134ae1e1e375b69a4d25242"
version = "v1.0.1"
[[projects]]
digest = "1:961dc3b1d11f969370533390fdf203813162980c858e1dabe827b60940c909a5"
name = "github.com/magiconair/properties"
packages = ["."]
pruneopts = ""
revision = "c2353362d570a7bfa228149c62842019201cfb71"
version = "v1.8.0"
[[projects]]
digest = "1:bcc46a0fbd9e933087bef394871256b5c60269575bb661935874729c65bbbf60"
name = "github.com/mitchellh/mapstructure"
packages = ["."]
pruneopts = ""
revision = "3536a929edddb9a5b34bd6861dc4a9647cb459fe"
version = "v1.1.2"
[[projects]]
digest = "1:0a57933aaf17a56940a600fbbe6ede0e0fcc92531d7b85a29f2a6352c12573ca"
name = "github.com/paypal/gorealis"
packages = [
".",
"gen-go/apache/aurora",
"response",
]
pruneopts = ""
revision = "b776bd301d9018b86253711e789eb2376ce60d1c"
version = "v2.0.0"
[[projects]]
digest = "1:894aef961c056b6d85d12bac890bf60c44e99b46292888bfa66caf529f804457"
name = "github.com/pelletier/go-toml"
packages = ["."]
pruneopts = ""
revision = "c01d1270ff3e442a8a57cddc1c92dc1138598194"
version = "v1.2.0"
[[projects]]
digest = "1:df48fb76fb2a40edea0c9b3d960bc95e326660d82ff1114e1f88001f7a236b40"
name = "github.com/pkg/errors"
packages = ["."]
pruneopts = ""
revision = "e881fd58d78e04cf6d0de1217f8707c8cc2249bc"
[[projects]]
digest = "1:78bea5e26e82826dacc5fd64a1013a6711b7075ec8072819b89e6ad76cb8196d"
name = "github.com/samuel/go-zookeeper"
packages = ["zk"]
pruneopts = ""
revision = "471cd4e61d7a78ece1791fa5faa0345dc8c7d5a5"
[[projects]]
digest = "1:9d57e200ef5ccc4217fe0a34287308bac652435e7c6513f6263e0493d2245c56"
name = "github.com/sirupsen/logrus"
packages = ["."]
pruneopts = ""
revision = "bcd833dfe83d3cebad139e4a29ed79cb2318bf95"
version = "v1.2.0"
[[projects]]
digest = "1:d0431c2fd72e39ee43ea7742322abbc200c3e704c9102c5c3c2e2e667095b0ca"
name = "github.com/spf13/afero"
packages = [
".",
"mem",
]
pruneopts = ""
revision = "d40851caa0d747393da1ffb28f7f9d8b4eeffebd"
version = "v1.1.2"
[[projects]]
digest = "1:d0b38ba6da419a6d4380700218eeec8623841d44a856bb57369c172fbf692ab4"
name = "github.com/spf13/cast"
packages = ["."]
pruneopts = ""
revision = "8965335b8c7107321228e3e3702cab9832751bac"
version = "v1.2.0"
[[projects]]
digest = "1:9ba49264cef4386aded205f9cb5b1f2d30f983d7dc37a21c780d9db3edfac9a7"
name = "github.com/spf13/cobra"
packages = ["."]
pruneopts = ""
revision = "fe5e611709b0c57fa4a89136deaa8e1d4004d053"
[[projects]]
digest = "1:9ceffa4ab5f7195ecf18b3a7fff90c837a9ed5e22e66d18069e4bccfe1f52aa0"
name = "github.com/spf13/jwalterweatherman"
packages = ["."]
pruneopts = ""
revision = "4a4406e478ca629068e7768fc33f3f044173c0a6"
version = "v1.0.0"
[[projects]]
digest = "1:cbaf13cdbfef0e4734ed8a7504f57fe893d471d62a35b982bf6fb3f036449a66"
name = "github.com/spf13/pflag"
packages = ["."]
pruneopts = ""
revision = "298182f68c66c05229eb03ac171abe6e309ee79a"
version = "v1.0.3"
[[projects]]
digest = "1:c08585b9af4c0f3c516af8e06254a66b462aee46937d41b7d8434795051e33b6"
name = "github.com/spf13/viper"
packages = ["."]
pruneopts = ""
revision = "62edee319679b6ceaec16de03b966102d2dea709"
[[projects]]
branch = "master"
digest = "1:efb97ebbd73c3a7068579327f5d98a17c09f6d8caf1fa3212c506e8bb78126b5"
name = "golang.org/x/crypto"
packages = ["ssh/terminal"]
pruneopts = ""
revision = "4d3f4d9ffa16a13f451c3b2999e9c49e9750bf06"
[[projects]]
branch = "master"
digest = "1:3d1785223d1b588249f131ddb1a88b1f80ee3eaa20934aba6562a44e633bbe7d"
name = "golang.org/x/sys"
packages = [
"unix",
"windows",
]
pruneopts = ""
revision = "44b849a8bc13eb42e95e6c6c5e360481b73ec710"
[[projects]]
digest = "1:5acd3512b047305d49e8763eef7ba423901e85d5dd2fd1e71778a0ea8de10bd4"
name = "golang.org/x/text"
packages = [
"internal/gen",
"internal/triegen",
"internal/ucd",
"transform",
"unicode/cldr",
"unicode/norm",
]
pruneopts = ""
revision = "f21a4dfb5e38f5895301dc265a8def02365cc3d0"
version = "v0.3.0"
[[projects]]
digest = "1:f0620375dd1f6251d9973b5f2596228cc8042e887cd7f827e4220bc1ce8c30e2"
name = "gopkg.in/yaml.v2"
packages = ["."]
pruneopts = ""
revision = "5420a8b6744d3b0345ab293f6fcba19c978f1183"
version = "v2.2.1"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"git.apache.org/thrift.git/lib/go/thrift",
"github.com/paypal/gorealis",
"github.com/paypal/gorealis/gen-go/apache/aurora",
"github.com/pkg/errors",
"github.com/sirupsen/logrus",
"github.com/spf13/cobra",
"github.com/spf13/pflag",
"github.com/spf13/viper",
]
solver-name = "gps-cdcl"
solver-version = 1

View file

@ -1,17 +0,0 @@
required = ["git.apache.org/thrift.git/lib/go/thrift"]
[[constraint]]
name = "github.com/paypal/gorealis"
version = ">=2.0.0"
[[constraint]]
name = "github.com/spf13/cobra"
revision = "fe5e611709b0c57fa4a89136deaa8e1d4004d053"
[[constraint]]
name = "github.com/spf13/viper"
revision = "62edee319679b6ceaec16de03b966102d2dea709"
[[constraint]]
name = "git.apache.org/thrift.git"
branch = "0.12.0"

View file

@ -1,12 +1,12 @@
# Australis
A light-weight client for [Apache Aurora](https://aurora.apache.org/) built using [gorealis](https://github.com/paypal/gorealis).
A light-weight client for [Aurora Scheduler](https://aurora-scheduler.github.io/) built using [gorealis](https://github.com/aurora-scheduler/gorealis).
## Usage
See the [documentation](docs/australis.md) for more information.
## Status
Australis is a work in progress and does not support all the features of Apache Aurora.
Australis is a work in progress and does not support all the features of Aurora Scheduler.
### Build locally
This project uses go mods. To build locally run:
@ -14,4 +14,4 @@ This project uses go mods. To build locally run:
`$ go build -o australis main.go`
### Building debian package
From the inside of the deb-packaging folder, run [build_deb.sh](deb-packaging/build_deb.sh)
From the inside of the deb-packaging folder, run [build_deb.sh](deb-packaging/build_deb.sh)

View file

@ -42,4 +42,3 @@ In MacOS this directory is $(brew --prefix)/etc/bash_completion.d if auto comple
rootCmd.GenBashCompletionFile(filename)
},
}

View file

@ -15,188 +15,47 @@
package cmd
import (
"os"
"strings"
yaml "gopkg.in/yaml.v2"
realis "github.com/paypal/gorealis/v2"
"github.com/pkg/errors"
"github.com/aurora-scheduler/australis/internal"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(createCmd)
createCmd.Flags().BoolVarP(&monitor, "monitor", "m", true, "monitor the result after sending the command")
}
var createCmd = &cobra.Command{
Use: "create",
Short: "Create an Aurora Job",
Run: createJob,
Args: cobra.ExactArgs(1),
}
type URI struct {
URI string `yaml:"uri"`
Extract bool `yaml:"extract"`
Cache bool `yaml:"cache"`
}
type Executor struct {
Name string `yaml:"name"`
Data string `yaml:"data"`
}
type ThermosProcess struct {
Name string `yaml:"name"`
Cmd string `yaml:"cmd"`
}
type DockerContainer struct {
Name string `yaml:"name"`
Tag string `yaml:"tag"`
}
type Container struct {
Docker *DockerContainer `yaml:"docker"`
}
type Job struct {
Environment string `yaml:"environment"`
Role string `yaml:"role"`
Name string `yaml:"name"`
CPU float64 `yaml:"cpu"`
RAM int64 `yaml:"ram"`
Disk int64 `yaml:"disk"`
Executor Executor `yaml:"executor"`
Instances int32 `yaml:"instances"`
URIs []URI `yaml:"uris"`
Metadata map[string]string `yaml:"labels"`
Service bool `yaml:"service"`
Thermos []ThermosProcess `yaml:",flow,omitempty"`
Container *Container `yaml:"container,omitempty"`
}
func (j *Job) Validate() bool {
if j.Name == "" {
return false
}
if j.Role == "" {
return false
}
if j.Environment == "" {
return false
}
if j.Instances <= 0 {
return false
}
if j.CPU <= 0.0 {
return false
}
if j.RAM <= 0 {
return false
}
if j.Disk <= 0 {
return false
}
return true
}
func unmarshalJob(filename string) (Job, error) {
job := Job{}
if jobsFile, err := os.Open(filename); err != nil {
return job, errors.Wrap(err, "unable to read the job config file")
} else {
if err := yaml.NewDecoder(jobsFile).Decode(&job); err != nil {
return job, errors.Wrap(err, "unable to parse job config file")
}
if !job.Validate() {
return job, errors.New("invalid job config")
}
}
return job, nil
Args: cobra.RangeArgs(1, 2),
}
func createJob(cmd *cobra.Command, args []string) {
job, err := unmarshalJob(args[0])
job, err := internal.UnmarshalJob(args[0])
if err != nil {
log.Fatalln(err)
}
auroraJob := realis.NewJob().
Environment(job.Environment).
Role(job.Role).
Name(job.Name).
CPU(job.CPU).
RAM(job.RAM).
Disk(job.Disk).
IsService(job.Service).
InstanceCount(job.Instances)
// Adding URIs.
for _, uri := range job.URIs {
auroraJob.AddURIs(uri.Extract, uri.Cache, uri.URI)
}
// Adding Metadata.
for key, value := range job.Metadata {
auroraJob.AddLabel(key, value)
}
// If thermos jobs processes are provided, use them
if len(job.Thermos) > 0 {
thermosExec := realis.ThermosExecutor{}
for _, process := range job.Thermos {
thermosExec.AddProcess(realis.NewThermosProcess(process.Name, process.Cmd))
}
auroraJob.ThermosExecutor(thermosExec)
} else if job.Executor.Name != "" {
// Non-Thermos executor
if job.Executor.Name == "" {
log.Fatal("no executor name provided")
}
auroraJob.ExecutorName(job.Executor.Name)
auroraJob.ExecutorData(job.Executor.Data)
} else if job.Container != nil {
if job.Container.Docker == nil {
log.Fatal("no container specified")
}
if job.Container.Docker.Tag != "" && !strings.ContainsRune(job.Container.Docker.Name, ':') {
job.Container.Docker.Name += ":" + job.Container.Docker.Tag
}
auroraJob.Container(realis.NewDockerContainer().Image(job.Container.Docker.Name))
} else {
log.Fatal("task does not contain a thermos definition, a custom executor name, or a container to launch")
auroraJob, err := job.ToRealis()
if err != nil {
log.Fatalln(err)
}
if err := client.CreateJob(auroraJob); err != nil {
log.Fatal("unable to create Aurora job: ", err)
}
if ok, monitorErr := client.MonitorInstances(auroraJob.JobKey(),
auroraJob.GetInstanceCount(),
5,
50); !ok || monitorErr != nil {
if err := client.KillJob(auroraJob.JobKey()); err != nil {
log.Fatal(monitorErr, err)
if monitor {
if ok, monitorErr := client.MonitorInstances(auroraJob.JobKey(),
auroraJob.GetInstanceCount(),
5,
50); !ok || monitorErr != nil {
if err := client.KillJob(auroraJob.JobKey()); err != nil {
log.Fatal(monitorErr, err)
}
log.Fatal(monitorErr)
}
log.Fatal(monitorErr)
}
}

View file

@ -15,14 +15,33 @@
package cmd
import (
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"net/http"
"strings"
realis "github.com/paypal/gorealis/v2"
"github.com/paypal/gorealis/v2/gen-go/apache/aurora"
"github.com/aurora-scheduler/australis/internal"
realis "github.com/aurora-scheduler/gorealis/v2"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
)
const (
localAgentStateURL = "http://127.0.0.1:5051/state"
)
type mesosAgentState struct {
Flags mesosAgentFlags `json:"flags,omitempty"`
}
type mesosAgentFlags struct {
Master string `json:"master,omitempty"`
hasMaster bool // indicates if the master flag contains direct Master's address
}
func init() {
rootCmd.AddCommand(fetchCmd)
@ -61,12 +80,100 @@ func init() {
help(cmd, s)
})
mesosLeaderCmd.Flags().String("zkPath", "/mesos", "Zookeeper node path where mesos leader election happens")
mesosCmd.AddCommand(mesosLeaderCmd)
fetchCmd.AddCommand(mesosCmd)
// Hijack help function to hide unnecessary global flags
mesosCmd.SetHelpFunc(func(cmd *cobra.Command, s []string) {
if cmd.HasInheritedFlags() {
cmd.InheritedFlags().VisitAll(func(f *pflag.Flag) {
if f.Name != "logLevel" {
f.Hidden = true
}
})
}
help(cmd, s)
})
/* Fetch Master nodes/Leader */
masterCmd.Flags().String("zkPath", "/aurora/scheduler", "Zookeeper node path to get master nodes/leader")
fetchCmd.AddCommand(masterCmd)
// Hijack help function to hide unnecessary global flags
masterCmd.SetHelpFunc(func(cmd *cobra.Command, s []string) {
if cmd.HasInheritedFlags() {
cmd.InheritedFlags().VisitAll(func(f *pflag.Flag) {
if f.Name != "logLevel" {
f.Hidden = true
}
})
}
help(cmd, s)
})
mesosMasterCmd.Flags().String("zkPath", "/mesos", "Zookeeper node path to get mesos master nodes/leader")
mesosCmd.AddCommand(mesosMasterCmd)
// Hijack help function to hide unnecessary global flags
mesosMasterCmd.SetHelpFunc(func(cmd *cobra.Command, s []string) {
if cmd.HasInheritedFlags() {
cmd.InheritedFlags().VisitAll(func(f *pflag.Flag) {
if f.Name != "logLevel" {
f.Hidden = true
}
})
}
help(cmd, s)
})
// Fetch jobs
fetchJobsCmd.Flags().StringVarP(role, "role", "r", "", "Aurora Role")
fetchCmd.AddCommand(fetchJobsCmd)
// Fetch Status
fetchCmd.AddCommand(fetchStatusCmd)
// fetch quota
fetchCmd.AddCommand(fetchQuotaCmd)
// fetch capacity
fetchCmd.AddCommand(fetchAvailCapacityCmd)
// Hijack help function to hide unnecessary global flags
fetchAvailCapacityCmd.SetHelpFunc(func(cmd *cobra.Command, s []string) {
if cmd.HasInheritedFlags() {
cmd.InheritedFlags().VisitAll(func(f *pflag.Flag) {
if f.Name != "logLevel" {
f.Hidden = true
}
})
}
help(cmd, s)
})
// fetch tasks with status
fetchCmd.AddCommand(fetchTasksWithStatusCmd)
fetchTasksWithStatusCmd.Flags().StringVarP(taskStatus, "status", "x", "", "Task Status")
fetchTasksWithStatusCmd.MarkFlagRequired("status")
fetchTasksWithStatusCmd.Flags().StringVarP(env, "environment", "e", "", "Aurora Environment")
fetchTasksWithStatusCmd.Flags().StringVarP(role, "role", "r", "", "Aurora Role")
fetchTasksWithStatusCmd.Flags().StringVarP(name, "name", "n", "", "Aurora Name")
// Hijack help function to hide unnecessary global flags
fetchTasksWithStatusCmd.SetHelpFunc(func(cmd *cobra.Command, s []string) {
if cmd.HasInheritedFlags() {
cmd.InheritedFlags().VisitAll(func(f *pflag.Flag) {
if f.Name != "logLevel" {
f.Hidden = true
}
})
}
help(cmd, s)
})
}
var fetchCmd = &cobra.Command{
@ -95,8 +202,8 @@ var taskStatusCmd = &cobra.Command{
var leaderCmd = &cobra.Command{
Use: "leader [zkNode0, zkNode1, ...zkNodeN]",
PersistentPreRun: func(cmd *cobra.Command, args []string) {}, //We don't need a realis client for this cmd
PersistentPostRun: func(cmd *cobra.Command, args []string) {}, //We don't need a realis client for this cmd
PersistentPreRun: func(cmd *cobra.Command, args []string) {}, // We don't need a realis client for this cmd
PersistentPostRun: func(cmd *cobra.Command, args []string) {}, // We don't need a realis client for this cmd
PreRun: setConfig,
Args: cobra.MinimumNArgs(1),
Short: "Fetch current Aurora leader given Zookeeper nodes. ",
@ -105,6 +212,48 @@ Pass Zookeeper nodes separated by a space as an argument to this command.`,
Run: fetchLeader,
}
var masterCmd = &cobra.Command{
Use: "master [zkNode0 zkNode1 ...zkNodeN]",
PersistentPreRun: func(cmd *cobra.Command, args []string) {}, // We don't need a realis client for this cmd
PersistentPostRun: func(cmd *cobra.Command, args []string) {}, // We don't need a realis client for this cmd
PreRun: setConfig,
Args: cobra.MinimumNArgs(1),
Short: "Fetch current Aurora master nodes/leader given Zookeeper nodes. ",
Long: `Gets the current aurora master nodes/leader using information from Zookeeper path.
Pass Zookeeper nodes separated by a space as an argument to this command.`,
Run: fetchMaster,
}
var mesosCmd = &cobra.Command{
Use: "mesos",
PreRun: setConfig,
Short: "Fetch information from Mesos.",
}
var mesosLeaderCmd = &cobra.Command{
Use: "leader [zkNode0, zkNode1, ...zkNodeN]",
PersistentPreRun: func(cmd *cobra.Command, args []string) {}, // We don't need a realis client for this cmd
PersistentPostRun: func(cmd *cobra.Command, args []string) {}, // We don't need a realis client for this cmd
PreRun: setConfig,
Short: "Fetch current Mesos-master leader given Zookeeper nodes.",
Long: `Gets the current leading Mesos-master instance using information from Zookeeper path.
Pass Zookeeper nodes separated by a space as an argument to this command. If no nodes are provided,
it fetches leader from local Mesos agent or Zookeeper`,
Run: fetchMesosLeader,
}
var mesosMasterCmd = &cobra.Command{
Use: "master [zkNode0 zkNode1 ...zkNodeN]",
PersistentPreRun: func(cmd *cobra.Command, args []string) {}, // We don't need a realis client for this cmd
PersistentPostRun: func(cmd *cobra.Command, args []string) {}, // We don't need a realis client for this cmd
PreRun: setConfig,
Short: "Fetch current Mesos-master nodes/leader given Zookeeper nodes.",
Long: `Gets the current Mesos-master instances using information from Zookeeper path.
Pass Zookeeper nodes separated by a space as an argument to this command. If no nodes are provided,
it fetches Mesos-master nodes/leader from local Mesos agent or Zookeeper`,
Run: fetchMesosMaster,
}
var fetchJobsCmd = &cobra.Command{
Use: "jobs",
Short: "Fetch a list of task Aurora running under a role.",
@ -119,6 +268,28 @@ var fetchStatusCmd = &cobra.Command{
Run: fetchHostStatus,
}
var fetchQuotaCmd = &cobra.Command{
Use: "quota",
Short: "Fetch the quotas of given roles",
Long: `This command will print list of resource quotas with the aggregated resources for the given roles`,
Run: fetchQuota,
}
var fetchAvailCapacityCmd = &cobra.Command{
Use: "capacity",
PreRun: setConfig,
Short: "Fetch capacity report",
Long: `This command will show detailed capacity report of the cluster`,
Run: fetchAvailCapacity,
}
var fetchTasksWithStatusCmd = &cobra.Command{
Use: "tasks",
Short: "Fetch tasks with status",
Long: `This command will return the list of tasks with a given status`,
Run: fetchTasksWithStatus,
}
func fetchTasksConfig(cmd *cobra.Command, args []string) {
log.Infof("Fetching job configuration for [%s/%s/%s] \n", *env, *role, *name)
@ -138,11 +309,11 @@ func fetchTasksConfig(cmd *cobra.Command, args []string) {
tasks, err := client.GetTasksWithoutConfigs(taskQuery)
if err != nil {
log.Fatalf("error: %+v\n", err)
log.Fatalf("error: %+v", err)
}
if toJson {
fmt.Println(toJSON(tasks))
fmt.Println(internal.ToJSON(tasks))
} else {
for _, t := range tasks {
fmt.Println(t)
@ -164,7 +335,7 @@ func fetchTasksStatus(cmd *cobra.Command, args []string) {
if *role == "" {
role = nil
}
//TODO: Add filtering down by status
// TODO(rdelvalle): Add filtering down by status
taskQuery := &aurora.TaskQuery{
Environment: env,
Role: role,
@ -173,11 +344,11 @@ func fetchTasksStatus(cmd *cobra.Command, args []string) {
tasks, err := client.GetTaskStatus(taskQuery)
if err != nil {
log.Fatalf("error: %+v\n", err)
log.Fatalf("error: %+v", err)
}
if toJson {
fmt.Println(toJSON(tasks))
fmt.Println(internal.ToJSON(tasks))
} else {
for _, t := range tasks {
fmt.Println(t)
@ -193,7 +364,7 @@ func fetchHostStatus(cmd *cobra.Command, args []string) {
}
if toJson {
fmt.Println(toJSON(result.Statuses))
fmt.Println(internal.ToJSON(result.Statuses))
} else {
for _, k := range result.GetStatuses() {
fmt.Printf("Result: %s:%s\n", k.Host, k.Mode)
@ -217,6 +388,135 @@ func fetchLeader(cmd *cobra.Command, args []string) {
fmt.Println(url)
}
func fetchMesosLeader(cmd *cobra.Command, args []string) {
if len(args) < 1 {
mesosAgentFlags, err := fetchMasterFromAgent(localAgentStateURL)
if err != nil || mesosAgentFlags.Master == "" {
log.Debugf("unable to fetch Mesos leader via local Mesos agent: %v", err)
args = append(args, "localhost")
} else if mesosAgentFlags.hasMaster {
fmt.Println(mesosAgentFlags.Master)
return
} else {
args = append(args, strings.Split(mesosAgentFlags.Master, ",")...)
}
}
log.Infof("Fetching Mesos-master leader from Zookeeper node(s): %v \n", args)
url, err := realis.MesosFromZKOpts(realis.ZKEndpoints(args...), realis.ZKPath(cmd.Flag("zkPath").Value.String()))
if err != nil {
log.Fatalf("error: %+v\n", err)
}
fmt.Println(url)
}
func fetchMaster(cmd *cobra.Command, args []string) {
log.Infof("Fetching master nodes from %v \n", args)
if len(args) < 1 {
log.Fatalln("At least one Zookeeper node address must be passed in.")
}
masterMap, err := realis.MasterNodesFromZKOpts(realis.ZKEndpoints(args...), realis.ZKPath(cmd.Flag("zkPath").Value.String()))
if err != nil {
log.Fatalf("error: %+v\n", err)
}
if toJson {
fmt.Println(internal.ToJSON(masterMap))
} else {
for key, masterNodes := range masterMap {
for _, masterNode := range masterNodes {
fmt.Println(key + "=" + masterNode)
}
}
}
}
func fetchMesosMaster(cmd *cobra.Command, args []string) {
if len(args) < 1 {
mesosAgentFlags, err := fetchMasterFromAgent(localAgentStateURL)
if err != nil || mesosAgentFlags.Master == "" {
log.Debugf("unable to fetch Mesos master nodes via local Mesos agent: %v", err)
args = append(args, "localhost")
} else {
args = append(args, strings.Split(mesosAgentFlags.Master, ",")...)
}
}
log.Infof("Fetching Mesos-master nodes from Zookeeper node(s): %v \n", args)
mesosMasterMap, err := realis.MesosMasterNodesFromZKOpts(realis.ZKEndpoints(args...), realis.ZKPath(cmd.Flag("zkPath").Value.String()))
if err != nil {
log.Fatalf("error: %+v\n", err)
}
if toJson {
fmt.Println(internal.ToJSON(mesosMasterMap))
} else {
for key, mesosMasterNodes := range mesosMasterMap {
for _, mesosMasterNode := range mesosMasterNodes {
fmt.Println(key + "=" + mesosMasterNode)
}
}
}
}
func fetchMasterFromAgent(url string) (mesosAgentFlags mesosAgentFlags, err error) {
resp, err := http.Get(url)
if err != nil {
return
}
if resp.StatusCode != 200 {
return
}
defer resp.Body.Close()
state := &mesosAgentState{}
err = json.NewDecoder(resp.Body).Decode(state)
if err != nil {
return
}
mesosAgentFlags = state.Flags
err = updateMasterFlag(&mesosAgentFlags)
return
}
/*
Master flag can be passed as one of :
host:port
zk://host1:port1,host2:port2,.../path
zk://username:password@host1:port1,host2:port2,.../path
file:///path/to/file
This function takes care of all the above cases and updates flags with parsed values
*/
func updateMasterFlag(flags *mesosAgentFlags) error {
zkPathPrefix := "zk://"
filePathPrefix := "file://"
if strings.HasPrefix(flags.Master, zkPathPrefix) {
beginIndex := len(zkPathPrefix)
if strings.Contains(flags.Master, "@") {
beginIndex = strings.Index(flags.Master, "@") + 1
}
flags.Master = flags.Master[beginIndex:strings.LastIndex(flags.Master, "/")]
} else if strings.HasPrefix(flags.Master, filePathPrefix) {
content, err := ioutil.ReadFile(flags.Master)
if err != nil {
return err
}
if strings.Contains(string(content), filePathPrefix) {
return errors.New("invalid master file content")
}
flags.Master = string(content)
return updateMasterFlag(flags)
} else {
flags.hasMaster = true
}
return nil
}
// TODO: Expand this to be able to filter by job name and environment.
func fetchJobs(cmd *cobra.Command, args []string) {
log.Infof("Fetching tasks under role: %s \n", *role)
@ -233,7 +533,7 @@ func fetchJobs(cmd *cobra.Command, args []string) {
result, err := client.GetJobs(*role)
if err != nil {
log.Fatalf("error: %+v\n", err)
log.Fatalf("error: %+v", err)
}
if toJson {
@ -243,10 +543,143 @@ func fetchJobs(cmd *cobra.Command, args []string) {
configSlice = append(configSlice, config)
}
fmt.Println(toJSON(configSlice))
fmt.Println(internal.ToJSON(configSlice))
} else {
for jobConfig := range result.GetConfigs() {
fmt.Println(jobConfig)
}
}
}
//fetchQuota gets quotas for roles in args
func fetchQuota(cmd *cobra.Command, args []string) {
for _, role := range args {
log.Infof("Fetching quota for role: %s \n", role)
result, err := client.GetQuota(role)
if err != nil {
log.Fatalf("error: %+v\n", err)
}
if toJson {
fmt.Println(internal.ToJSON(result))
} else {
fmt.Printf(" Quota: %v\n", internal.ToJSON(result.Quota.GetResources()))
fmt.Printf(" Aggregated Resources: \n")
fmt.Printf(" ProdSharedConsumption: %v\n", internal.ToJSON(result.ProdSharedConsumption.GetResources()))
fmt.Printf(" NonProdSharedConsumption: %v\n",
internal.ToJSON(result.NonProdSharedConsumption.GetResources()))
fmt.Printf(" ProdDedicatedConsumption: %v\n",
internal.ToJSON(result.ProdDedicatedConsumption.GetResources()))
fmt.Printf(" NonProdDedicatedConsumption: %v\n",
internal.ToJSON(result.NonProdDedicatedConsumption.GetResources()))
}
}
}
//fetchAvailCapacity reports free capacity in details
func fetchAvailCapacity(cmd *cobra.Command, args []string) {
log.Infof("Fetching available capacity from %s/offers\n", client.GetSchedulerURL())
report, err := client.AvailOfferReport()
if err != nil {
log.Fatalf("error: %+v\n", err)
}
// convert report to user-friendly structure
capacity := map[string]map[string]map[string]int64{}
for g, gv := range report {
if _, ok := capacity[g]; !ok {
capacity[g] = map[string]map[string]int64{}
}
for r, rc := range gv {
if _, ok := capacity[g][r]; !ok {
capacity[g][r] = map[string]int64{}
}
for v, c := range rc {
capacity[g][r][fmt.Sprint(v)] = c
}
}
}
if toJson {
fmt.Println(internal.ToJSON(capacity))
if err != nil {
log.Fatalf("error: %+v\n", err)
}
} else {
fmt.Println(capacity)
}
}
//fetchTasksWithStatus returns lists of tasks for a given set of status
func fetchTasksWithStatus(cmd *cobra.Command, args []string) {
status := *taskStatus
log.Infof("Fetching tasks for role/environment/job:[%s/%s/%s] \n", *role, *env, *name)
log.Infof("Fetching tasks for a given status: %v \n", status)
// This Query takes nil for values it shouldn't need to match against.
// This allows us to potentially avoid expensive calls for specific environments, roles, or job names.
if *env == "" {
env = nil
}
if *role == "" {
role = nil
}
if *name == "" {
name = nil
}
// role needs to be specified if env is specified
if env != nil {
if role == nil {
log.Fatalln("Role must be specified when env is specified.")
}
}
// role or env needs to be specified if name is specified
if name != nil {
if role == nil && env == nil {
log.Fatalln("Role or env must be specified when name is specified.")
}
}
queryStatuses, err := scheduleStatusFromString(status)
if err != nil {
log.Fatalf("error: %+v", err)
}
taskQuery := &aurora.TaskQuery{Environment: env, Role: role, JobName: name, Statuses: queryStatuses}
tasks, err := client.GetTasksWithoutConfigs(taskQuery)
if err != nil {
log.Fatalf("error: %+v", err)
}
if toJson {
taskStatus := strings.ToUpper(status)
// convert task lists to a list of task id like role-env-name-[instance-id]
taskIdsMap := map[string][]string{}
var taskIds []string
for _, task := range tasks {
taskIds = append(taskIds, task.AssignedTask.TaskId)
}
taskIdsMap[taskStatus] = taskIds
fmt.Println(internal.ToJSON(taskIdsMap))
} else {
fmt.Printf("Tasks for status %s:\n", strings.ToUpper(status))
for _, t := range tasks {
fmt.Println(t.AssignedTask.TaskId)
}
}
}
// Convert status slice into ScheduleStatus slice
func scheduleStatusFromString(status string) ([]aurora.ScheduleStatus, error) {
scheduleStatus, err := aurora.ScheduleStatusFromString(strings.ToUpper(status))
if err != nil {
return nil, err
}
result := []aurora.ScheduleStatus{scheduleStatus}
return result, nil
}

View file

@ -69,7 +69,7 @@ func backup(cmd *cobra.Command, args []string) {
fmt.Println("Forcing scheduler to write a Backup of latest Snapshot to file system")
err := client.PerformBackup()
if err != nil {
log.Fatalf("error: %+v\n", err)
log.Fatalf("error: %+v", err)
} else {
log.Println("Backup started successfully")
}
@ -112,7 +112,7 @@ func explicitRecon(cmd *cobra.Command, args []string) {
// Get batch size from args and convert it to the right format
batchInt, err := strconv.Atoi(args[0])
if err != nil {
log.Fatalf("error: %+v\n", err)
log.Fatalf("error: %v", err)
}
batchInt32 := int32(batchInt)
@ -123,7 +123,7 @@ func explicitRecon(cmd *cobra.Command, args []string) {
err := client.ForceExplicitTaskReconciliation(batchSize)
if err != nil {
log.Fatalf("error: %+v\n", err.Error())
log.Fatalf("error: %v", err)
} else {
fmt.Println("Explicit reconciliation started successfully")
}
@ -142,7 +142,7 @@ func implicitRecon(cmd *cobra.Command, args []string) {
log.Println("Forcing scheduler to perform an implicit reconciliation with Mesos")
err := client.ForceImplicitTaskReconciliation()
if err != nil {
log.Fatalf("error: %+v\n", err)
log.Fatalf("error: %+v", err)
} else {
fmt.Println("Implicit reconciliation started successfully")
}

View file

@ -15,7 +15,10 @@
package cmd
import (
realis "github.com/paypal/gorealis/v2"
"strconv"
"strings"
realis "github.com/aurora-scheduler/gorealis/v2"
"github.com/spf13/cobra"
)
@ -26,13 +29,26 @@ func init() {
// Kill Job
killCmd.AddCommand(killJobCmd)
killCmd.AddCommand(killTasksCmd)
killJobCmd.Flags().StringVarP(env, "environment", "e", "", "Aurora Environment")
killJobCmd.Flags().StringVarP(role, "role", "r", "", "Aurora Role")
killJobCmd.Flags().StringVarP(name, "name", "n", "", "Aurora Name")
killJobCmd.Flags().BoolVarP(&monitor, "monitor", "m", true, "monitor the result after sending the command")
killJobCmd.MarkFlagRequired("environment")
killJobCmd.MarkFlagRequired("role")
killJobCmd.MarkFlagRequired("name")
//Set flags for killTask sub-command
killTasksCmd.Flags().StringVarP(env, "environment", "e", "", "Aurora Environment")
killTasksCmd.Flags().StringVarP(role, "role", "r", "", "Aurora Role")
killTasksCmd.Flags().StringVarP(name, "name", "n", "", "Aurora Name")
killTasksCmd.Flags().StringVarP(instances, "instances", "I", "", "Instances e.g. 1, 2, 5")
killTasksCmd.Flags().BoolVarP(&monitor, "monitor", "m", true, "monitor the result after sending the command")
killTasksCmd.MarkFlagRequired("environment")
killTasksCmd.MarkFlagRequired("role")
killTasksCmd.MarkFlagRequired("name")
killTasksCmd.MarkFlagRequired("instances")
}
var killCmd = &cobra.Command{
@ -46,6 +62,24 @@ var killJobCmd = &cobra.Command{
Run: killJob,
}
/*
* The killTasks command allows the user to kill a specific task of a job.
* The command also allows the user to kill multiple tasks of the same job. To do so the user needs to pass a list of instance numbers as comma separated values.
* Pass the instance number of the job to be killed after the --instances or -I flag
* Please note that all the instances passed must belong to the same job.
*
* example : australis kill tasks -e "environment" -r "role" -n "job_name" -I "1"
* The above example kills instance number 1.
*
* example 2 : australis kill tasks -e "environment" -r "role" -n "job_name" -I "1, 5, 9"
* The above example kills tasks 1, 5 and 9, which are part of the same job
*/
var killTasksCmd = &cobra.Command{
Use: "tasks",
Short: "Kill Aurora Tasks",
Run: killTasks,
}
func killJob(cmd *cobra.Command, args []string) {
log.Infof("Killing job [Env:%s Role:%s Name:%s]\n", *env, *role, *name)
@ -57,8 +91,49 @@ func killJob(cmd *cobra.Command, args []string) {
if err != nil {
log.Fatalln(err)
}
if ok, err := client.MonitorInstances(job.JobKey(), 0, 5, 50); !ok || err != nil {
log.Fatalln("Unable to kill all instances of job")
if monitor {
if ok, err := client.MonitorInstances(job.JobKey(), 0, 5, 50); !ok || err != nil {
log.Fatalln("Unable to kill all instances of job")
}
}
}
func killTasks(cmd *cobra.Command, args []string) {
log.Infof("Killing task [Env:%s Role:%s Name:%s Instance:%s]\n", *env, *role, *name, *instances)
//Set jobKey for the tasks to be killed.
task := realis.NewTask().
Environment(*env).
Role(*role).
Name(*name)
/*
* In the following block, we convert instance numbers, which were passed as strings, to integer values
* After converting them to integers, we add them to a slice of type int32.
*/
splitString := strings.Split(*instances, ",")
instanceList := make([]int32, len(splitString))
for i := range instanceList {
splitString[i] = strings.TrimSpace(splitString[i])
instanceNumber, intErr := strconv.Atoi(splitString[i])
if intErr != nil {
log.Fatalln("Instance passed should be a number. Error: " + intErr.Error())
return
} else {
instanceList[i] = int32(instanceNumber)
}
}
//Call the killtasks function, passing the instanceList as the list of instances to be killed.
if _, err := client.KillInstances(task.JobKey(), instanceList...); err != nil {
log.Fatalln(err)
}
if monitor {
if ok, err := client.MonitorInstances(task.JobKey(), 0, 5, 50); !ok || err != nil {
log.Fatalln("Unable to kill the given task")
}
}
}

View file

@ -18,19 +18,20 @@ import (
"strings"
"time"
"github.com/paypal/gorealis/v2/gen-go/apache/aurora"
"github.com/aurora-scheduler/australis/internal"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(monitorCmd)
monitorCmd.AddCommand(monitorHostCmd.cmd)
monitorCmd.AddCommand(monitorHostCmd.Cmd)
monitorHostCmd.cmd.Run = monitorHost
monitorHostCmd.cmd.Flags().DurationVar(&monitorHostCmd.monitorInterval, "interval", time.Second*5, "Interval at which to poll scheduler.")
monitorHostCmd.cmd.Flags().DurationVar(&monitorHostCmd.monitorTimeout, "timeout", time.Minute*10, "Time after which the monitor will stop polling and throw an error.")
monitorHostCmd.cmd.Flags().StringSliceVar(&monitorHostCmd.statusList, "statuses", []string{aurora.MaintenanceMode_DRAINED.String()}, "List of acceptable statuses for a host to be in. (case-insensitive) [NONE, SCHEDULED, DRAINED, DRAINING]")
monitorHostCmd.Cmd.Run = monitorHost
monitorHostCmd.Cmd.Flags().DurationVar(&monitorHostCmd.MonitorInterval, "interval", time.Second*5, "Interval at which to poll scheduler.")
monitorHostCmd.Cmd.Flags().DurationVar(&monitorHostCmd.MonitorTimeout, "timeout", time.Minute*10, "Time after which the monitor will stop polling and throw an error.")
monitorHostCmd.Cmd.Flags().StringSliceVar(&monitorHostCmd.StatusList, "statuses", []string{aurora.MaintenanceMode_DRAINED.String()}, "List of acceptable statuses for a host to be in. (case-insensitive) [NONE, SCHEDULED, DRAINED, DRAINING]")
}
var monitorCmd = &cobra.Command{
@ -38,20 +39,20 @@ var monitorCmd = &cobra.Command{
Short: "Watch for a specific state change",
}
var monitorHostCmd = monitorCmdConfig{
cmd: &cobra.Command{
var monitorHostCmd = internal.MonitorCmdConfig{
Cmd: &cobra.Command{
Use: "hosts",
Short: "Watch a host maintenance status until it enters one of the desired statuses.",
Long: `Provide a list of hosts to monitor for desired statuses. Statuses may be passed using the --statuses
flag with a list of comma separated statuses. Statuses include [NONE, SCHEDULED, DRAINED, DRAINING]`,
},
statusList: make([]string, 0),
StatusList: make([]string, 0),
}
func monitorHost(cmd *cobra.Command, args []string) {
maintenanceModes := make([]aurora.MaintenanceMode, 0)
for _, status := range monitorHostCmd.statusList {
for _, status := range monitorHostCmd.StatusList {
mode, err := aurora.MaintenanceModeFromString(strings.ToUpper(status))
if err != nil {
log.Fatal(err)
@ -60,10 +61,10 @@ func monitorHost(cmd *cobra.Command, args []string) {
maintenanceModes = append(maintenanceModes, mode)
}
log.Infof("Monitoring for %v at %v intervals", monitorHostCmd.monitorTimeout, monitorHostCmd.monitorInterval)
hostResult, err := client.MonitorHostMaintenance(args, maintenanceModes, monitorHostCmd.monitorInterval, monitorHostCmd.monitorTimeout)
log.Infof("Monitoring for %v at %v intervals", monitorHostCmd.MonitorTimeout, monitorHostCmd.MonitorInterval)
hostResult, err := client.MonitorHostMaintenance(args, maintenanceModes, monitorHostCmd.MonitorInterval, monitorHostCmd.MonitorTimeout)
maintenanceMonitorPrint(hostResult, maintenanceModes)
internal.MaintenanceMonitorPrint(hostResult, maintenanceModes, toJson)
if err != nil {
log.Fatal(err)

View file

@ -15,7 +15,7 @@
package cmd
import (
"github.com/paypal/gorealis/v2/gen-go/apache/aurora"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
"github.com/spf13/cobra"
)

View file

@ -15,34 +15,96 @@
package cmd
import (
"github.com/paypal/gorealis/v2/gen-go/apache/aurora"
"github.com/spf13/cobra"
"strconv"
"strings"
realis "github.com/aurora-scheduler/gorealis/v2"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(restartCmd)
rootCmd.AddCommand(restartCmd)
restartCmd.AddCommand(restartJobCmd)
restartJobCmd.Flags().StringVarP(env, "environment", "e", "", "Aurora Environment")
restartJobCmd.Flags().StringVarP(role, "role", "r", "", "Aurora Role")
restartJobCmd.Flags().StringVarP(name, "name", "n", "", "Aurora Name")
restartCmd.AddCommand(restartJobCmd)
restartJobCmd.Flags().StringVarP(env, "environment", "e", "", "Aurora Environment")
restartJobCmd.Flags().StringVarP(role, "role", "r", "", "Aurora Role")
restartJobCmd.Flags().StringVarP(name, "name", "n", "", "Aurora Name")
restartCmd.AddCommand(restartTasksCmd)
restartTasksCmd.Flags().StringVarP(env, "environment", "e", "", "Aurora Environment")
restartTasksCmd.Flags().StringVarP(role, "role", "r", "", "Aurora Role")
restartTasksCmd.Flags().StringVarP(name, "name", "n", "", "Aurora Name")
restartTasksCmd.Flags().StringVarP(instances, "instances", "I", "", "Instances e.g. 1, 2, 5")
restartTasksCmd.Flags().BoolVarP(&monitor, "monitor", "m", true, "monitor the result after sending the command")
restartTasksCmd.MarkFlagRequired("environment")
restartTasksCmd.MarkFlagRequired("role")
restartTasksCmd.MarkFlagRequired("name")
restartTasksCmd.MarkFlagRequired("instances")
}
var restartCmd = &cobra.Command{
Use: "restart",
Short: "Restart an Aurora Job.",
Use: "restart",
Short: "Restart an Aurora Job.",
}
var restartJobCmd = &cobra.Command{
Use: "job",
Short: "Restart a Job.",
Run: restartJob,
Use: "job",
Short: "Restart a Job.",
Run: restartJob,
}
var restartTasksCmd = &cobra.Command{
Use: "tasks",
Short: "Restart tasks for a Job.",
Run: restartTasks,
}
func restartJob(cmd *cobra.Command, args []string) {
key := aurora.JobKey{Environment: *env, Role: *role, Name: *name}
if err := client.RestartJob(key); err != nil {
log.Fatal("unable to create Aurora job: ", err)
}
key := aurora.JobKey{Environment: *env, Role: *role, Name: *name}
if err := client.RestartJob(key); err != nil {
log.Fatal("unable to create Aurora job: ", err)
}
}
func restartTasks(cmd *cobra.Command, args []string) {
log.Infof("Restarts task [Env:%s Role:%s Name:%s Instance:%s Monitor:%s]\n", *env, *role, *name, *instances, strconv.FormatBool(monitor))
//Set jobKey for the tasks to be killed.
task := realis.NewTask().
Environment(*env).
Role(*role).
Name(*name)
/*
* In the following block, we convert instance numbers, which were passed as strings, to integer values
* After converting them to integers, we add them to a slice of type int32.
*/
splitString := strings.Split(*instances, ",")
instanceList := make([]int32, len(splitString))
for i := range instanceList {
splitString[i] = strings.TrimSpace(splitString[i])
var instanceNumber int
var err error
if instanceNumber, err = strconv.Atoi(splitString[i]); err != nil {
log.Fatalln("Instance passed should be a number. Error: " + err.Error())
return
}
instanceList[i] = int32(instanceNumber)
}
//Call the RestartInstances function, passing the instanceList as the list of instances to be restarted.
if err := client.RestartInstances(task.JobKey(), instanceList...); err != nil {
log.Fatalln(err)
}
if monitor {
if ok, err := client.MonitorInstances(task.JobKey(), int32(len(instanceList)), 5, 50); !ok || err != nil {
log.Fatalln("Monitor failed to monitor the given task after restart. Error: " + err.Error())
}
}
}

View file

@ -15,7 +15,7 @@
package cmd
import (
"github.com/paypal/gorealis/v2/gen-go/apache/aurora"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
"github.com/spf13/cobra"
)

View file

@ -17,7 +17,7 @@ package cmd
import (
"fmt"
"github.com/paypal/gorealis/v2/gen-go/apache/aurora"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
"github.com/spf13/cobra"
)

View file

@ -18,9 +18,10 @@ import (
"strings"
"time"
"github.com/aurora-scheduler/australis/internal"
"github.com/spf13/viper"
realis "github.com/paypal/gorealis/v2"
realis "github.com/aurora-scheduler/gorealis/v2"
"github.com/spf13/cobra"
"github.com/sirupsen/logrus"
@ -28,7 +29,8 @@ import (
var username, password, zkAddr, schedAddr string
var env, role, name = new(string), new(string), new(string)
var ram, disk int64
var dedicated string
var ram, disk, gpu, port int64
var cpu float64
var client *realis.Client
var skipCertVerification bool
@ -45,9 +47,13 @@ var count int64
var filename string
var message = new(string)
var updateID string
var monitor bool
var timeout time.Duration
var log = logrus.New()
var taskStatus = new(string)
var instances = new(string)
const australisVer = "v0.1.0"
const australisVer = "v1.0.5"
var forceDrainTimeout time.Duration
@ -65,7 +71,8 @@ func init() {
rootCmd.PersistentFlags().BoolVarP(&skipCertVerification, "skipCertVerification", "i", false, "Skip CA certificate hostname verification.")
rootCmd.PersistentFlags().StringVar(&configFile, "config", "/etc/aurora/australis.yml", "Config file to use.")
rootCmd.PersistentFlags().BoolVar(&toJson, "toJSON", false, "Print output in JSON format.")
rootCmd.PersistentFlags().StringVarP(&logLevel, "logLevel", "l", "info", "Set logging level ["+getLoggingLevels()+"].")
rootCmd.PersistentFlags().StringVarP(&logLevel, "logLevel", "l", "info", "Set logging level ["+internal.GetLoggingLevels()+"].")
rootCmd.PersistentFlags().DurationVarP(&timeout, "timeout", "t", 20*time.Second, "Gorealis timeout.")
}
var rootCmd = &cobra.Command{
@ -91,10 +98,11 @@ func setConfig(cmd *cobra.Command, args []string) {
lvl, err := logrus.ParseLevel(logLevel)
if err != nil {
log.Fatalf("Log level %v is not valid\n", logLevel)
log.Fatalf("Log level %v is not valid", logLevel)
}
log.SetLevel(lvl)
internal.Logger(log)
}
func connect(cmd *cobra.Command, args []string) {
@ -139,7 +147,7 @@ func connect(cmd *cobra.Command, args []string) {
realisOptions := []realis.ClientOption{realis.BasicAuth(username, password),
realis.ThriftJSON(),
realis.Timeout(20 * time.Second),
realis.Timeout(timeout),
realis.BackOff(realis.Backoff{
Steps: 2,
Duration: 10 * time.Second,

52
cmd/schedule.go Normal file
View file

@ -0,0 +1,52 @@
/**
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package cmd
import (
"github.com/aurora-scheduler/australis/internal"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(scheduleCmd)
}
var scheduleCmd = &cobra.Command{
Use: "schedule",
Short: "Schedule a cron job on Aurora scheduler",
Run: scheduleCron,
Args: cobra.ExactArgs(1),
}
func scheduleCron(cmd *cobra.Command, args []string) {
job, err := internal.UnmarshalJob(args[0])
if err != nil {
log.Fatalln(err)
}
if err := job.ValidateCron(); err != nil {
log.Fatal(err)
}
auroraJob, err := job.ToRealis()
if err != nil {
log.Fatalln(err)
}
if err := client.ScheduleCronJob(auroraJob); err != nil {
log.Fatal("unable to schedule job: ", err)
}
}

61
cmd/simulate.go Normal file
View file

@ -0,0 +1,61 @@
/**
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package cmd
import (
"fmt"
"github.com/aurora-scheduler/australis/internal"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(simulateCmd)
simulateCmd.AddCommand(fitCmd)
}
var simulateCmd = &cobra.Command{
Use: "simulate",
Short: "Simulate some work based on the current cluster condition, and return the output",
}
var fitCmd = &cobra.Command{
Use: "fit",
Short: "Compute how many tasks can we fit to a cluster",
Run: fit,
Args: cobra.RangeArgs(1, 2),
}
func fit(cmd *cobra.Command, args []string) {
log.Infof("Compute how many tasks can be fit in the remaining cluster capacity")
taskConfig, err := internal.UnmarshalTaskConfig(args[0])
if err != nil {
log.Fatalln(err)
}
offers, err := client.Offers()
if err != nil {
log.Fatal("error: %+v", err)
}
numTasks, err := client.FitTasks(taskConfig, offers)
if err != nil {
log.Fatal("error: %+v", err)
}
fmt.Println(numTasks)
}

View file

@ -21,7 +21,8 @@ import (
"os"
"time"
"github.com/paypal/gorealis/v2/gen-go/apache/aurora"
"github.com/aurora-scheduler/australis/internal"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
"github.com/spf13/cobra"
)
@ -34,46 +35,52 @@ func init() {
rootCmd.AddCommand(startCmd)
// Sub-commands
startCmd.AddCommand(startDrainCmd.cmd)
startDrainCmd.cmd.Run = drain
startCmd.AddCommand(startDrainCmd.Cmd)
startDrainCmd.Cmd.Run = drain
// Maintenance specific flags
startDrainCmd.cmd.Flags().DurationVar(&startDrainCmd.monitorInterval, "interval", time.Second*5, "Interval at which to poll scheduler.")
startDrainCmd.cmd.Flags().DurationVar(&startDrainCmd.monitorTimeout, "timeout", time.Minute*10, "Time after which the monitor will stop polling and throw an error.")
startDrainCmd.cmd.Flags().StringVar(&fromJsonFile, jsonFileFlag, "", "JSON file to read list of agents from.")
startDrainCmd.cmd.Flags().BoolVar(&fromJson, jsonFlag, false, "Read JSON list of agents from the STDIN.")
startDrainCmd.Cmd.Flags().DurationVar(&startDrainCmd.MonitorInterval, "interval", time.Second*5, "Interval at which to poll scheduler.")
startDrainCmd.Cmd.Flags().DurationVar(&startDrainCmd.MonitorTimeout, "timeout", time.Minute*10, "Time after which the monitor will stop polling and throw an error.")
startDrainCmd.Cmd.Flags().StringVar(&fromJsonFile, jsonFileFlag, "", "JSON file to read list of agents from.")
startDrainCmd.Cmd.Flags().BoolVar(&fromJson, jsonFlag, false, "Read JSON list of agents from the STDIN.")
/* SLA Aware commands */
startCmd.AddCommand(startSLADrainCmd.cmd)
startSLADrainCmd.cmd.Run = slaDrain
startCmd.AddCommand(startSLADrainCmd.Cmd)
startSLADrainCmd.Cmd.Run = slaDrain
// SLA Maintenance specific flags
startSLADrainCmd.cmd.Flags().Int64Var(&count, countFlag, 5, "Instances count that should be running to meet SLA.")
startSLADrainCmd.cmd.Flags().Float64Var(&percent, percentageFlag, 80.0, "Percentage of instances that should be running to meet SLA.")
startSLADrainCmd.cmd.Flags().DurationVar(&duration, "duration", time.Minute*1, "Minimum time duration a task needs to be `RUNNING` to be treated as active.")
startSLADrainCmd.cmd.Flags().DurationVar(&forceDrainTimeout, "sla-limit", time.Minute*60, "Time limit after which SLA-Aware drain sheds SLA Awareness.")
startSLADrainCmd.cmd.Flags().DurationVar(&startSLADrainCmd.monitorInterval, "interval", time.Second*10, "Interval at which to poll scheduler.")
startSLADrainCmd.cmd.Flags().DurationVar(&startSLADrainCmd.monitorTimeout, "timeout", time.Minute*20, "Time after which the monitor will stop polling and throw an error.")
startSLADrainCmd.cmd.Flags().StringVar(&fromJsonFile, jsonFileFlag, "", "JSON file to read list of agents from.")
startSLADrainCmd.cmd.Flags().BoolVar(&fromJson, jsonFlag, false, "Read JSON list of agents from the STDIN.")
startSLADrainCmd.Cmd.Flags().Int64Var(&count, countFlag, 5, "Instances count that should be running to meet SLA.")
startSLADrainCmd.Cmd.Flags().Float64Var(&percent, percentageFlag, 80.0, "Percentage of instances that should be running to meet SLA.")
startSLADrainCmd.Cmd.Flags().DurationVar(&duration, "duration", time.Minute*1, "Minimum time duration a task needs to be `RUNNING` to be treated as active.")
startSLADrainCmd.Cmd.Flags().DurationVar(&forceDrainTimeout, "sla-limit", time.Minute*60, "Time limit after which SLA-Aware drain sheds SLA Awareness.")
startSLADrainCmd.Cmd.Flags().DurationVar(&startSLADrainCmd.MonitorInterval, "interval", time.Second*10, "Interval at which to poll scheduler.")
startSLADrainCmd.Cmd.Flags().DurationVar(&startSLADrainCmd.MonitorTimeout, "timeout", time.Minute*20, "Time after which the monitor will stop polling and throw an error.")
startSLADrainCmd.Cmd.Flags().StringVar(&fromJsonFile, jsonFileFlag, "", "JSON file to read list of agents from.")
startSLADrainCmd.Cmd.Flags().BoolVar(&fromJson, jsonFlag, false, "Read JSON list of agents from the STDIN.")
startCmd.AddCommand(startMaintenanceCmd.cmd)
startMaintenanceCmd.cmd.Run = maintenance
startCmd.AddCommand(startMaintenanceCmd.Cmd)
startMaintenanceCmd.Cmd.Run = maintenance
// SLA Maintenance specific flags
startMaintenanceCmd.cmd.Flags().DurationVar(&startMaintenanceCmd.monitorInterval, "interval", time.Second*5, "Interval at which to poll scheduler.")
startMaintenanceCmd.cmd.Flags().DurationVar(&startMaintenanceCmd.monitorTimeout, "timeout", time.Minute*10, "Time after which the monitor will stop polling and throw an error.")
startMaintenanceCmd.cmd.Flags().StringVar(&fromJsonFile, jsonFileFlag, "", "JSON file to read list of agents from.")
startMaintenanceCmd.cmd.Flags().BoolVar(&fromJson, jsonFlag, false, "Read JSON list of agents from the STDIN.")
startMaintenanceCmd.Cmd.Flags().DurationVar(&startMaintenanceCmd.MonitorInterval, "interval", time.Second*5, "Interval at which to poll scheduler.")
startMaintenanceCmd.Cmd.Flags().DurationVar(&startMaintenanceCmd.MonitorTimeout, "timeout", time.Minute*10, "Time after which the monitor will stop polling and throw an error.")
startMaintenanceCmd.Cmd.Flags().StringVar(&fromJsonFile, jsonFileFlag, "", "JSON file to read list of agents from.")
startMaintenanceCmd.Cmd.Flags().BoolVar(&fromJson, jsonFlag, false, "Read JSON list of agents from the STDIN.")
// Start update command
startCmd.AddCommand(startUpdateCmd.Cmd)
startUpdateCmd.Cmd.Run = update
startUpdateCmd.Cmd.Flags().DurationVar(&startUpdateCmd.MonitorInterval, "interval", time.Second*5, "Interval at which to poll scheduler.")
startUpdateCmd.Cmd.Flags().DurationVar(&startUpdateCmd.MonitorTimeout, "timeout", time.Minute*10, "Time after which the monitor will stop polling and throw an error.")
}
var startCmd = &cobra.Command{
Use: "start",
Short: "Start a service, maintenance on a host (DRAIN), a snapshot, or a backup.",
Short: "Start a service, maintenance on a host (DRAIN), a snapshot, an update, or a backup.",
}
var startDrainCmd = monitorCmdConfig{
cmd: &cobra.Command{
var startDrainCmd = internal.MonitorCmdConfig{
Cmd: &cobra.Command{
Use: "drain [space separated host list or use JSON flags]",
Short: "Place a list of space separated Mesos Agents into draining mode.",
Long: `Adds a Mesos Agent to Aurora's Drain list. Agents in this list
@ -84,8 +91,8 @@ expects a space separated list of hosts to place into maintenance mode.`,
},
}
var startSLADrainCmd = monitorCmdConfig{
cmd: &cobra.Command{
var startSLADrainCmd = internal.MonitorCmdConfig{
Cmd: &cobra.Command{
Use: "sla-drain [space separated host list or use JSON flags]",
Short: "Place a list of space separated Mesos Agents into maintenance mode using SLA aware strategies.",
Long: `Adds a Mesos Agent to Aurora's Drain list. Agents in this list
@ -100,8 +107,8 @@ when a Job does not have a defined SLA policy.`,
},
}
var startMaintenanceCmd = monitorCmdConfig{
cmd: &cobra.Command{
var startMaintenanceCmd = internal.MonitorCmdConfig{
Cmd: &cobra.Command{
Use: "maintenance [space separated host list or use JSON flags]",
Short: "Place a list of space separated Mesos Agents into maintenance mode.",
Long: `Places Mesos Agent into Maintenance mode. Agents in this list
@ -111,6 +118,16 @@ expects a space separated list of hosts to place into maintenance mode.`,
},
}
var startUpdateCmd = internal.MonitorCmdConfig{
Cmd: &cobra.Command{
Use: "update [update config]",
Short: "Start an update on an Aurora long running service.",
Long: `Starts the update process on an Aurora long running service. If no such service exists, the update mechanism
will act as a deployment, creating all new instances based on the requirements in the update configuration.`,
Args: cobra.ExactArgs(1),
},
}
func argsValidateJSONFlags(cmd *cobra.Command, args []string) error {
if cmd.Flags().Changed(jsonFlag) && cmd.Flags().Changed(jsonFileFlag) {
return errors.New("only json file or json stdin must be set")
@ -162,15 +179,15 @@ func drain(cmd *cobra.Command, args []string) {
log.Debugln(result)
log.Infof("Monitoring for %v at %v intervals", monitorHostCmd.monitorTimeout, monitorHostCmd.monitorInterval)
log.Infof("Monitoring for %v at %v intervals", monitorHostCmd.MonitorTimeout, monitorHostCmd.MonitorInterval)
// Monitor change to DRAINING and DRAINED mode
hostResult, err := client.MonitorHostMaintenance(
hosts,
[]aurora.MaintenanceMode{aurora.MaintenanceMode_DRAINED},
startDrainCmd.monitorInterval,
startDrainCmd.monitorTimeout)
startDrainCmd.MonitorInterval,
startDrainCmd.MonitorTimeout)
maintenanceMonitorPrint(hostResult, []aurora.MaintenanceMode{aurora.MaintenanceMode_DRAINED})
internal.MaintenanceMonitorPrint(hostResult, []aurora.MaintenanceMode{aurora.MaintenanceMode_DRAINED}, toJson)
if err != nil {
log.Fatalln(err)
@ -180,7 +197,7 @@ func drain(cmd *cobra.Command, args []string) {
func slaDrainHosts(policy *aurora.SlaPolicy, interval, timeout time.Duration, hosts ...string) {
result, err := client.SLADrainHosts(policy, int64(forceDrainTimeout.Seconds()), hosts...)
if err != nil {
log.Fatalf("error: %+v\n", err)
log.Fatalf("error: %+v", err)
}
log.Debugln(result)
@ -193,7 +210,7 @@ func slaDrainHosts(policy *aurora.SlaPolicy, interval, timeout time.Duration, ho
interval,
timeout)
maintenanceMonitorPrint(hostResult, []aurora.MaintenanceMode{aurora.MaintenanceMode_DRAINED})
internal.MaintenanceMonitorPrint(hostResult, []aurora.MaintenanceMode{aurora.MaintenanceMode_DRAINED}, toJson)
if err != nil {
log.Fatalf("error: %+v", err)
@ -213,7 +230,7 @@ func slaDrain(cmd *cobra.Command, args []string) {
if cmd.Flags().Changed(percentageFlag) {
log.Infoln("Setting hosts to DRAINING with the Percentage SLA policy.")
policy.PercentageSlaPolicy = &aurora.PercentageSlaPolicy{
Percentage: percent,
Percentage: percent,
DurationSecs: int64(duration.Seconds()),
}
}
@ -224,7 +241,7 @@ func slaDrain(cmd *cobra.Command, args []string) {
}
log.Infoln("Hosts affected: ", args)
slaDrainHosts(policy, startDrainCmd.monitorInterval, startDrainCmd.monitorTimeout, hosts...)
slaDrainHosts(policy, startDrainCmd.MonitorInterval, startDrainCmd.MonitorTimeout, hosts...)
}
func maintenance(cmd *cobra.Command, args []string) {
@ -239,18 +256,42 @@ func maintenance(cmd *cobra.Command, args []string) {
log.Debugln(result)
log.Infof("Monitoring for %v at %v intervals", monitorHostCmd.monitorTimeout, monitorHostCmd.monitorInterval)
log.Infof("Monitoring for %v at %v intervals", monitorHostCmd.MonitorTimeout, monitorHostCmd.MonitorInterval)
// Monitor change to DRAINING and DRAINED mode
hostResult, err := client.MonitorHostMaintenance(
hosts,
[]aurora.MaintenanceMode{aurora.MaintenanceMode_SCHEDULED},
startMaintenanceCmd.monitorInterval,
startMaintenanceCmd.monitorTimeout)
startMaintenanceCmd.MonitorInterval,
startMaintenanceCmd.MonitorTimeout)
maintenanceMonitorPrint(hostResult, []aurora.MaintenanceMode{aurora.MaintenanceMode_SCHEDULED})
internal.MaintenanceMonitorPrint(hostResult, []aurora.MaintenanceMode{aurora.MaintenanceMode_SCHEDULED}, toJson)
if err != nil {
log.Fatalln("error: %+v", err)
log.Fatalf("error: %+v", err)
}
}
func update(cmd *cobra.Command, args []string) {
updateJob, err := internal.UnmarshalUpdate(args[0])
if err != nil {
log.Fatal(err)
}
update, err := updateJob.ToRealis()
if err != nil {
log.Fatal(err)
}
result, err := client.StartJobUpdate(update, "")
if err != nil {
log.Fatalf("Update failed to start %v", err)
}
if ok, monitorErr := client.MonitorJobUpdate(*result.GetKey(),
startUpdateCmd.MonitorInterval,
startUpdateCmd.MonitorTimeout); !ok || monitorErr != nil {
log.Fatal("update did not ROLL FORWARD before monitor timed out")
}
}

View file

@ -17,20 +17,19 @@ package cmd
import (
"time"
"github.com/paypal/gorealis/v2/gen-go/apache/aurora"
"github.com/aurora-scheduler/australis/internal"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
"github.com/spf13/cobra"
)
var stopMaintenanceConfig = monitorCmdConfig{}
func init() {
rootCmd.AddCommand(stopCmd)
// Stop subcommands
stopCmd.AddCommand(stopMaintCmd.cmd)
stopMaintCmd.cmd.Run = endMaintenance
stopMaintCmd.cmd.Flags().DurationVar(&stopMaintenanceConfig.monitorInterval, "interval", time.Second*5, "Interval at which to poll scheduler.")
stopMaintCmd.cmd.Flags().DurationVar(&stopMaintenanceConfig.monitorTimeout, "timeout", time.Minute*1, "Time after which the monitor will stop polling and throw an error.")
stopCmd.AddCommand(stopMaintCmd.Cmd)
stopMaintCmd.Cmd.Run = endMaintenance
stopMaintCmd.Cmd.Flags().DurationVar(&stopMaintCmd.MonitorInterval, "interval", time.Second*5, "Interval at which to poll scheduler.")
stopMaintCmd.Cmd.Flags().DurationVar(&stopMaintCmd.MonitorTimeout, "timeout", time.Minute*1, "Time after which the monitor will stop polling and throw an error.")
// Stop update
@ -46,8 +45,8 @@ var stopCmd = &cobra.Command{
Short: "Stop a service or maintenance on a host (DRAIN).",
}
var stopMaintCmd = monitorCmdConfig{
cmd: &cobra.Command{
var stopMaintCmd = internal.MonitorCmdConfig{
Cmd: &cobra.Command{
Use: "drain [space separated host list]",
Short: "Stop maintenance on a host (move to NONE).",
Long: `Transition a list of hosts currently in a maintenance status out of it.`,
@ -75,13 +74,13 @@ func endMaintenance(cmd *cobra.Command, args []string) {
hostResult, err := client.MonitorHostMaintenance(
args,
[]aurora.MaintenanceMode{aurora.MaintenanceMode_NONE},
stopMaintenanceConfig.monitorInterval,
stopMaintenanceConfig.monitorTimeout)
stopMaintCmd.MonitorInterval,
stopMaintCmd.MonitorTimeout)
maintenanceMonitorPrint(hostResult, []aurora.MaintenanceMode{aurora.MaintenanceMode_NONE})
internal.MaintenanceMonitorPrint(hostResult, []aurora.MaintenanceMode{aurora.MaintenanceMode_NONE}, toJson)
if err != nil {
log.Fatalln("error: %+v", err)
log.Fatalf("error: %+v", err)
}
}

View file

@ -1,92 +0,0 @@
/**
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package cmd
import (
"bytes"
"encoding/json"
"fmt"
"time"
"github.com/paypal/gorealis/v2/gen-go/apache/aurora"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
type monitorCmdConfig struct {
cmd *cobra.Command
monitorInterval, monitorTimeout time.Duration
statusList []string
}
func toJSON(v interface{}) string {
output, err := json.Marshal(v)
if err != nil {
log.Fatalln("Unable to serialize Aurora response: %+v", v)
}
return string(output)
}
func getLoggingLevels() string {
var buffer bytes.Buffer
for _, level := range logrus.AllLevels {
buffer.WriteString(level.String())
buffer.WriteString(" ")
}
buffer.Truncate(buffer.Len() - 1)
return buffer.String()
}
func maintenanceMonitorPrint(hostResult map[string]bool, desiredStates []aurora.MaintenanceMode) {
if len(hostResult) > 0 {
// Create anonymous struct for JSON formatting
output := struct {
DesiredStates []string `json:desired_states`
Transitioned []string `json:transitioned`
NonTransitioned []string `json:non-transitioned`
}{
make([]string, 0),
make([]string, 0),
make([]string, 0),
}
for _, state := range desiredStates {
output.DesiredStates = append(output.DesiredStates, state.String())
}
for host, ok := range hostResult {
if ok {
output.Transitioned = append(output.Transitioned, host)
} else {
output.NonTransitioned = append(output.NonTransitioned, host)
}
}
if toJson {
fmt.Println(toJSON(output))
} else {
fmt.Printf("Entered %v status: %v\n", output.DesiredStates, output.Transitioned)
fmt.Printf("Did not enter %v status: %v\n", output.DesiredStates, output.NonTransitioned)
}
}
}

View file

@ -2,7 +2,7 @@ FROM ubuntu:16.04
RUN apt-get update -y && \
apt-get install -y build-essential devscripts dh-exec dh-make git lintian wget && \
wget https://dl.google.com/go/go1.12.6.linux-amd64.tar.gz -O /tmp/go.tar.gz
wget https://dl.google.com/go/go1.15.2.linux-amd64.tar.gz -O /tmp/go.tar.gz
RUN tar -C /usr/local -xzf /tmp/go.tar.gz

View file

@ -2,4 +2,4 @@
docker build . -t australis_deb_builder
docker run --rm -v $HOME/go/pkg/mod:/go/pkg/mod -v $(pwd)/..:/australis australis_builder
docker run --rm -v $HOME/go/pkg/mod:/go/pkg/mod -v $(pwd)/..:/australis australis_deb_builder

View file

@ -1,12 +1,9 @@
#!/bin/bash
# Temporary fix for a go mods bug
rm /australis/go.sum
# Build debian package
cd /australis
debuild -d -us -uc -b
# Move resulting packages to the dist folder
mkdir -p /australis/dist
mv /australis_*_amd64* /australis/dist
mv /australis_*_amd64* /australis/dist

57
debian/changelog vendored
View file

@ -1,3 +1,60 @@
australis (1.0.5) stable; urgency=medium
* kill tasks
* fetch all aurora & mesos master nodes
-- Nhat Tan Le <maintainer@nhatle.xyz> Wed, 31 Aug 2022 15:03:24 -0700
australis (1.0.4) stable; urgency=medium
* fetch free capacity
* simulate task fitting - compute how many tasks can be fit in the remaining capacity
-- Nhat Tan Le <maintainer@nhatle.xyz> Tue, 2 Aug 2022 16:19:24 -0700
australis (1.0.3) stable; urgency=medium
* Update CI to compile on PRs
* Add tier and production in task config
* Add fetch quota command
* Add priority into job config
-- Nhat Tan Le <maintainer@nhatle.xyz> Wed, 20 Oct 2021 14:24:10 -0700
australis (1.0.2) stable; urgency=medium
* Fixing broken Thrift dependency by bumping up backported version to thrift v0.13.2
-- Renan Del Valle <maintainer@ridv.xyz> Mon, 11 Jan 2021 17:57:10 -0800
australis (1.0.1) stable; urgency=medium
* Added flag -m and --monitor that can be set in order to monitor a job creation or a job kill. By default monitor is set to true.
-- Renan Del Valle <maintainer@ridv.xyz> Wed, 30 Sep 2020 16:46:56 -0700
australis (1.0.0) stable; urgency=medium
* First stable release.
-- Renan Del Valle <maintainer@ridv.xyz> Wed, 30 Sep 2020 15:39:29 -0700
australis (0.22.0) unstable; urgency=medium
* Added support for starting job updates.
* Added support for setting SlaAwareness for updates.
* Added upport for scheduling cron jobs.
-- Renan Del Valle <maintainer@ridv.xyz> Thu, 07 May 2020 12:00:00 -0700
australis (0.1.1) unstable; urgency=medium
* Removed support for building with dep
* Upgraded depdencies to their latest versions.
* gorealis v2 now lives in the aurora-scheduler organization and dependencies have been updated to reflect that.
-- Renan DelValle <renanidelvalle@gmail.com> Mon, 19 Feb 2020 12:00:00 -0700
australis (0.1.0) unstable; urgency=medium
* Adding support for drain sub-command to take in JSON list from stdin or from a specified file.

2
debian/control vendored
View file

@ -3,7 +3,7 @@ Priority: optional
Maintainer: Renan DelValle <renanidelvalle@gmail.com>
Section: utils
Standards-Version: 4.2.1.3
Homepage: https://github.com/rdelval/australis
Homepage: https://github.com/aurora-scheduler/australis
Package: australis
Architecture: any

View file

@ -18,6 +18,7 @@ A light-weight command line client for use with Apache Aurora built using goreal
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -25,16 +26,19 @@ A light-weight command line client for use with Apache Aurora built using goreal
### SEE ALSO
* [australis autocomplete](australis_autocomplete.md) - Create auto completion for bash.
* [australis create](australis_create.md) - Create an Aurora Job
* [australis docs](australis_docs.md) - Kill an Aurora Job
* [australis fetch](australis_fetch.md) - Fetch information from Aurora
* [australis force](australis_force.md) - Force the scheduler to do a snapshot, a backup, or a task reconciliation.
* [australis kill](australis_kill.md) - Kill an Aurora Job
* [australis monitor](australis_monitor.md) - Watch for a specific state change
* [australis rollback](australis_rollback.md) - rollback an operation such as an Update
* [australis pulse](australis_pulse.md) - Pulse a Job update
* [australis restart](australis_restart.md) - Restart an Aurora Job.
* [australis resume](australis_resume.md) - Resume a Job update
* [australis rollback](australis_rollback.md) - Rollback an operation such as an Update
* [australis schedule](australis_schedule.md) - Schedule a cron job on Aurora scheduler
* [australis set](australis_set.md) - Set a value in the Aurora Scheduler.
* [australis start](australis_start.md) - Start a service, maintenance on a host (DRAIN), a snapshot, or a backup.
* [australis simulate](australis_simulate.md) - Simulate some work based on the current cluster condition, and return the output
* [australis start](australis_start.md) - Start a service, maintenance on a host (DRAIN), a snapshot, an update, or a backup.
* [australis stop](australis_stop.md) - Stop a service or maintenance on a host (DRAIN).
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -13,7 +13,8 @@ australis create [flags]
### Options
```
-h, --help help for create
-h, --help help for create
-m, --monitor monitor the result after sending the command (default true)
```
### Options inherited from parent commands
@ -27,6 +28,7 @@ australis create [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -36,4 +38,4 @@ australis create [flags]
* [australis](australis.md) - australis is a client for Apache Aurora
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -23,6 +23,7 @@ Fetch information from Aurora
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -31,9 +32,14 @@ Fetch information from Aurora
### SEE ALSO
* [australis](australis.md) - australis is a client for Apache Aurora
* [australis fetch capacity](australis_fetch_capacity.md) - Fetch capacity report
* [australis fetch jobs](australis_fetch_jobs.md) - Fetch a list of task Aurora running under a role.
* [australis fetch leader](australis_fetch_leader.md) - Fetch current Aurora leader given Zookeeper nodes.
* [australis fetch master](australis_fetch_master.md) - Fetch current Aurora master nodes/leader given Zookeeper nodes.
* [australis fetch mesos](australis_fetch_mesos.md) - Fetch information from Mesos.
* [australis fetch quota](australis_fetch_quota.md) - Fetch the quotas of given roles
* [australis fetch status](australis_fetch_status.md) - Fetch the maintenance status of a node from Aurora
* [australis fetch task](australis_fetch_task.md) - Task information from Aurora
* [australis fetch tasks](australis_fetch_tasks.md) - Fetch tasks with status
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,40 @@
## australis fetch capacity
Fetch capacity report
### Synopsis
This command will show detailed capacity report of the cluster
```
australis fetch capacity [flags]
```
### Options
```
-h, --help help for capacity
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis fetch](australis_fetch.md) - Fetch information from Aurora
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -28,6 +28,7 @@ australis fetch jobs [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -37,4 +38,4 @@ australis fetch jobs [flags]
* [australis fetch](australis_fetch.md) - Fetch information from Aurora
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -29,6 +29,7 @@ australis fetch leader [zkNode0, zkNode1, ...zkNodeN] [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -38,4 +39,4 @@ australis fetch leader [zkNode0, zkNode1, ...zkNodeN] [flags]
* [australis fetch](australis_fetch.md) - Fetch information from Aurora
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,42 @@
## australis fetch master
Fetch current Aurora master nodes/leader given Zookeeper nodes.
### Synopsis
Gets the current aurora master nodes/leader using information from Zookeeper path.
Pass Zookeeper nodes separated by a space as an argument to this command.
```
australis fetch master [zkNode0 zkNode1 ...zkNodeN] [flags]
```
### Options
```
-h, --help help for master
--zkPath string Zookeeper node path to get master nodes/leader (default "/aurora/scheduler")
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis fetch](australis_fetch.md) - Fetch information from Aurora
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,38 @@
## australis fetch mesos
Fetch information from Mesos.
### Synopsis
Fetch information from Mesos.
### Options
```
-h, --help help for mesos
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis fetch](australis_fetch.md) - Fetch information from Aurora
* [australis fetch mesos leader](australis_fetch_mesos_leader.md) - Fetch current Mesos-master leader given Zookeeper nodes.
* [australis fetch mesos master](australis_fetch_mesos_master.md) - Fetch current Mesos-master nodes/leader given Zookeeper nodes.
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,43 @@
## australis fetch mesos leader
Fetch current Mesos-master leader given Zookeeper nodes.
### Synopsis
Gets the current leading Mesos-master instance using information from Zookeeper path.
Pass Zookeeper nodes separated by a space as an argument to this command. If no nodes are provided,
it fetches leader from local Mesos agent or Zookeeper
```
australis fetch mesos leader [zkNode0, zkNode1, ...zkNodeN] [flags]
```
### Options
```
-h, --help help for leader
--zkPath string Zookeeper node path where mesos leader election happens (default "/mesos")
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis fetch mesos](australis_fetch_mesos.md) - Fetch information from Mesos.
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,43 @@
## australis fetch mesos master
Fetch current Mesos-master nodes/leader given Zookeeper nodes.
### Synopsis
Gets the current Mesos-master instances using information from Zookeeper path.
Pass Zookeeper nodes separated by a space as an argument to this command. If no nodes are provided,
it fetches Mesos-master nodes/leader from local Mesos agent or Zookeeper
```
australis fetch mesos master [zkNode0 zkNode1 ...zkNodeN] [flags]
```
### Options
```
-h, --help help for master
--zkPath string Zookeeper node path to get mesos master nodes/leader (default "/mesos")
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis fetch mesos](australis_fetch_mesos.md) - Fetch information from Mesos.
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,40 @@
## australis fetch quota
Fetch the quotas of given roles
### Synopsis
This command will print list of resource quotas with the aggregated resources for the given roles
```
australis fetch quota [flags]
```
### Options
```
-h, --help help for quota
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis fetch](australis_fetch.md) - Fetch information from Aurora
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -27,6 +27,7 @@ australis fetch status [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -36,4 +37,4 @@ australis fetch status [flags]
* [australis fetch](australis_fetch.md) - Fetch information from Aurora
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -23,6 +23,7 @@ Task information from Aurora
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -34,4 +35,4 @@ Task information from Aurora
* [australis fetch task config](australis_fetch_task_config.md) - Fetch a list of task configurations from Aurora.
* [australis fetch task status](australis_fetch_task_status.md) - Fetch task status for a Job key.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -30,6 +30,7 @@ australis fetch task config [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -39,4 +40,4 @@ australis fetch task config [flags]
* [australis fetch task](australis_fetch_task.md) - Task information from Aurora
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -30,6 +30,7 @@ australis fetch task status [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -39,4 +40,4 @@ australis fetch task status [flags]
* [australis fetch task](australis_fetch_task.md) - Task information from Aurora
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,44 @@
## australis fetch tasks
Fetch tasks with status
### Synopsis
This command will return the list of tasks with a given status
```
australis fetch tasks [flags]
```
### Options
```
-e, --environment string Aurora Environment
-h, --help help for tasks
-n, --name string Aurora Name
-r, --role string Aurora Role
-x, --status string Task Status
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis fetch](australis_fetch.md) - Fetch information from Aurora
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -23,6 +23,7 @@ Force the scheduler to do a snapshot, a backup, or a task reconciliation.
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -35,4 +36,4 @@ Force the scheduler to do a snapshot, a backup, or a task reconciliation.
* [australis force recon](australis_force_recon.md) - Force the leading scheduler to perform a reconciliation.
* [australis force snapshot](australis_force_snapshot.md) - Force the leading scheduler to perform a Snapshot.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -28,6 +28,7 @@ australis force backup [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -37,4 +38,4 @@ australis force backup [flags]
* [australis force](australis_force.md) - Force the scheduler to do a snapshot, a backup, or a task reconciliation.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -30,6 +30,7 @@ state for all currently known non-terminal tasks.
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -41,4 +42,4 @@ state for all currently known non-terminal tasks.
* [australis force recon explicit](australis_force_recon_explicit.md) - Force the leading scheduler to perform an explicit recon.
* [australis force recon implicit](australis_force_recon_implicit.md) - Force the leading scheduler to perform an implicit recon.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -29,6 +29,7 @@ australis force recon explicit [batch_size] [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -38,4 +39,4 @@ australis force recon explicit [batch_size] [flags]
* [australis force recon](australis_force_recon.md) - Force the leading scheduler to perform a reconciliation.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -28,6 +28,7 @@ australis force recon implicit [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -37,4 +38,4 @@ australis force recon implicit [flags]
* [australis force recon](australis_force_recon.md) - Force the leading scheduler to perform a reconciliation.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -28,6 +28,7 @@ australis force snapshot [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -37,4 +38,4 @@ australis force snapshot [flags]
* [australis force](australis_force.md) - Force the scheduler to do a snapshot, a backup, or a task reconciliation.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -23,6 +23,7 @@ Kill an Aurora Job
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -31,7 +32,7 @@ Kill an Aurora Job
### SEE ALSO
* [australis](australis.md) - australis is a client for Apache Aurora
* [australis kill entire-cluster](australis_kill_entire-cluster.md) - Kill every task in the cluster.
* [australis kill job](australis_kill_job.md) - Kill an Aurora Job
* [australis kill tasks](australis_kill_tasks.md) - Kill Aurora Tasks
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -15,6 +15,7 @@ australis kill job [flags]
```
-e, --environment string Aurora Environment
-h, --help help for job
-m, --monitor monitor the result after sending the command (default true)
-n, --name string Aurora Name
-r, --role string Aurora Role
```
@ -30,6 +31,7 @@ australis kill job [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -39,4 +41,4 @@ australis kill job [flags]
* [australis kill](australis_kill.md) - Kill an Aurora Job
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,45 @@
## australis kill tasks
Kill Aurora Tasks
### Synopsis
Kill Aurora Tasks
```
australis kill tasks [flags]
```
### Options
```
-e, --environment string Aurora Environment
-h, --help help for tasks
-I, --instances string Instances e.g. 1, 2, 5
-m, --monitor monitor the result after sending the command (default true)
-n, --name string Aurora Name
-r, --role string Aurora Role
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis kill](australis_kill.md) - Kill an Aurora Job
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -23,6 +23,7 @@ Watch for a specific state change
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -33,4 +34,4 @@ Watch for a specific state change
* [australis](australis.md) - australis is a client for Apache Aurora
* [australis monitor hosts](australis_monitor_hosts.md) - Watch a host maintenance status until it enters one of the desired statuses.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -17,7 +17,6 @@ australis monitor hosts [flags]
-h, --help help for hosts
--interval duration Interval at which to poll scheduler. (default 5s)
--statuses strings List of acceptable statuses for a host to be in. (case-insensitive) [NONE, SCHEDULED, DRAINED, DRAINING] (default [DRAINED])
--timeout duration Time after which the monitor will stop polling and throw an error. (default 10m0s)
```
### Options inherited from parent commands
@ -31,6 +30,7 @@ australis monitor hosts [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -40,4 +40,4 @@ australis monitor hosts [flags]
* [australis monitor](australis_monitor.md) - Watch for a specific state change
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

44
docs/australis_pulse.md Normal file
View file

@ -0,0 +1,44 @@
## australis pulse
Pulse a Job update
### Synopsis
Pulse a Job update
```
australis pulse [flags]
```
### Options
```
-e, --environment string Aurora Environment
-h, --help help for pulse
--id string Update ID
-n, --name string Aurora Name
-r, --role string Aurora Role
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis](australis.md) - australis is a client for Apache Aurora
###### Auto generated by spf13/cobra on 8-Sep-2022

37
docs/australis_restart.md Normal file
View file

@ -0,0 +1,37 @@
## australis restart
Restart an Aurora Job.
### Synopsis
Restart an Aurora Job.
### Options
```
-h, --help help for restart
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis](australis.md) - australis is a client for Apache Aurora
* [australis restart job](australis_restart_job.md) - Restart a Job.
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,43 @@
## australis restart job
Restart a Job.
### Synopsis
Restart a Job.
```
australis restart job [flags]
```
### Options
```
-e, --environment string Aurora Environment
-h, --help help for job
-n, --name string Aurora Name
-r, --role string Aurora Role
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis restart](australis_restart.md) - Restart an Aurora Job.
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,45 @@
## australis restart tasks
Restart tasks for a Job.
### Synopsis
Restart tasks for a Job.
```
australis restart tasks [flags]
```
### Options
```
-e, --environment string Aurora Environment
-h, --help help for tasks
-I, --instances string Instances e.g. 1, 2, 5
-m, --monitor monitor the result after sending the command (default true)
-n, --name string Aurora Name
-r, --role string Aurora Role
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis restart](australis_restart.md) - Restart an Aurora Job.
###### Auto generated by spf13/cobra on 21-Sep-2022

45
docs/australis_resume.md Normal file
View file

@ -0,0 +1,45 @@
## australis resume
Resume a Job update
### Synopsis
Resume a Job update
```
australis resume [flags]
```
### Options
```
-e, --environment string Aurora Environment
-h, --help help for resume
--id string Update ID
--message string Message to store along resume.
-n, --name string Aurora Name
-r, --role string Aurora Role
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis](australis.md) - australis is a client for Apache Aurora
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -1,10 +1,10 @@
## australis rollback
rollback an operation such as an Update
Rollback an operation such as an Update
### Synopsis
rollback an operation such as an Update
Rollback an operation such as an Update
### Options
@ -23,6 +23,7 @@ rollback an operation such as an Update
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -31,6 +32,6 @@ rollback an operation such as an Update
### SEE ALSO
* [australis](australis.md) - australis is a client for Apache Aurora
* [australis rollback update](australis_rollback_update.md) - rollback an update that is currently paused
* [australis rollback update](australis_rollback_update.md) - Rollback an update
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -1,10 +1,10 @@
## australis rollback update
rollback an update that is currently paused
Rollback an update
### Synopsis
rollback an update that is currently paused
Rollback an update
```
australis rollback update [flags]
@ -32,6 +32,7 @@ australis rollback update [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -39,6 +40,6 @@ australis rollback update [flags]
### SEE ALSO
* [australis rollback](australis_rollback.md) - rollback an operation such as an Update
* [australis rollback](australis_rollback.md) - Rollback an operation such as an Update
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -1,19 +1,19 @@
## australis kill entire-cluster
## australis schedule
Kill every task in the cluster.
Schedule a cron job on Aurora scheduler
### Synopsis
To be written.
Schedule a cron job on Aurora scheduler
```
australis kill entire-cluster [flags]
australis schedule [flags]
```
### Options
```
-h, --help help for entire-cluster
-h, --help help for schedule
```
### Options inherited from parent commands
@ -27,6 +27,7 @@ australis kill entire-cluster [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -34,6 +35,6 @@ australis kill entire-cluster [flags]
### SEE ALSO
* [australis kill](australis_kill.md) - Kill an Aurora Job
* [australis](australis.md) - australis is a client for Apache Aurora
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -23,6 +23,7 @@ Set a value in the Aurora Scheduler.
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -33,4 +34,4 @@ Set a value in the Aurora Scheduler.
* [australis](australis.md) - australis is a client for Apache Aurora
* [australis set quota](australis_set_quota.md) - Set Quota resources for a role.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -27,6 +27,7 @@ australis set quota <role> cpu:<value> ram:<value> disk:<value> [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -36,4 +37,4 @@ australis set quota <role> cpu:<value> ram:<value> disk:<value> [flags]
* [australis set](australis_set.md) - Set a value in the Aurora Scheduler.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,37 @@
## australis simulate
Simulate some work based on the current cluster condition, and return the output
### Synopsis
Simulate some work based on the current cluster condition, and return the output
### Options
```
-h, --help help for simulate
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis](australis.md) - australis is a client for Apache Aurora
* [australis simulate fit](australis_simulate_fit.md) - Compute how many tasks can we fit to a cluster
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -0,0 +1,40 @@
## australis simulate fit
Compute how many tasks can we fit to a cluster
### Synopsis
Compute how many tasks can we fit to a cluster
```
australis simulate fit [flags]
```
### Options
```
-h, --help help for fit
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis simulate](australis_simulate.md) - Simulate some work based on the current cluster condition, and return the output
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -1,10 +1,10 @@
## australis start
Start a service, maintenance on a host (DRAIN), a snapshot, or a backup.
Start a service, maintenance on a host (DRAIN), a snapshot, an update, or a backup.
### Synopsis
Start a service, maintenance on a host (DRAIN), a snapshot, or a backup.
Start a service, maintenance on a host (DRAIN), a snapshot, an update, or a backup.
### Options
@ -23,6 +23,7 @@ Start a service, maintenance on a host (DRAIN), a snapshot, or a backup.
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -34,5 +35,6 @@ Start a service, maintenance on a host (DRAIN), a snapshot, or a backup.
* [australis start drain](australis_start_drain.md) - Place a list of space separated Mesos Agents into draining mode.
* [australis start maintenance](australis_start_maintenance.md) - Place a list of space separated Mesos Agents into maintenance mode.
* [australis start sla-drain](australis_start_sla-drain.md) - Place a list of space separated Mesos Agents into maintenance mode using SLA aware strategies.
* [australis start update](australis_start_update.md) - Start an update on an Aurora long running service.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -10,7 +10,7 @@ are killed and rescheduled in an Agent that is not in maintenance mode. Command
expects a space separated list of hosts to place into maintenance mode.
```
australis start drain [space separated host list] [flags]
australis start drain [space separated host list or use JSON flags] [flags]
```
### Options
@ -18,7 +18,8 @@ australis start drain [space separated host list] [flags]
```
-h, --help help for drain
--interval duration Interval at which to poll scheduler. (default 5s)
--timeout duration Time after which the monitor will stop polling and throw an error. (default 10m0s)
--json Read JSON list of agents from the STDIN.
--json-file string JSON file to read list of agents from.
```
### Options inherited from parent commands
@ -32,6 +33,7 @@ australis start drain [space separated host list] [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -39,6 +41,6 @@ australis start drain [space separated host list] [flags]
### SEE ALSO
* [australis start](australis_start.md) - Start a service, maintenance on a host (DRAIN), a snapshot, or a backup.
* [australis start](australis_start.md) - Start a service, maintenance on a host (DRAIN), a snapshot, an update, or a backup.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -9,7 +9,7 @@ are de-prioritized for scheduling a task. Command
expects a space separated list of hosts to place into maintenance mode.
```
australis start maintenance [space separated host list] [flags]
australis start maintenance [space separated host list or use JSON flags] [flags]
```
### Options
@ -17,7 +17,8 @@ australis start maintenance [space separated host list] [flags]
```
-h, --help help for maintenance
--interval duration Interval at which to poll scheduler. (default 5s)
--timeout duration Time after which the monitor will stop polling and throw an error. (default 10m0s)
--json Read JSON list of agents from the STDIN.
--json-file string JSON file to read list of agents from.
```
### Options inherited from parent commands
@ -31,6 +32,7 @@ australis start maintenance [space separated host list] [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -38,6 +40,6 @@ australis start maintenance [space separated host list] [flags]
### SEE ALSO
* [australis start](australis_start.md) - Start a service, maintenance on a host (DRAIN), a snapshot, or a backup.
* [australis start](australis_start.md) - Start a service, maintenance on a host (DRAIN), a snapshot, an update, or a backup.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -8,11 +8,26 @@ Adds a Mesos Agent to Aurora's Drain list. Agents in this list
are not allowed to schedule new tasks and any tasks already running on this Agent
are killed and rescheduled in an Agent that is not in maintenance mode. Command
expects a space separated list of hosts to place into maintenance mode.
If the --count argument is passed, tasks will be drained using the count SLA policy as a fallback
when a Job does not have a defined SLA policy.
If the --percentage argument is passed, tasks will be drained using the percentage SLA policy as a fallback
when a Job does not have a defined SLA policy.
```
australis start sla-drain [space separated host list or use JSON flags] [flags]
```
### Options
```
-h, --help help for sla-drain
--count int Instances count that should be running to meet SLA. (default 5)
--duration RUNNING Minimum time duration a task needs to be RUNNING to be treated as active. (default 1m0s)
-h, --help help for sla-drain
--interval duration Interval at which to poll scheduler. (default 10s)
--json Read JSON list of agents from the STDIN.
--json-file string JSON file to read list of agents from.
--percentage float Percentage of instances that should be running to meet SLA. (default 80)
--sla-limit duration Time limit after which SLA-Aware drain sheds SLA Awareness. (default 1h0m0s)
```
### Options inherited from parent commands
@ -26,6 +41,7 @@ expects a space separated list of hosts to place into maintenance mode.
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -33,8 +49,6 @@ expects a space separated list of hosts to place into maintenance mode.
### SEE ALSO
* [australis start](australis_start.md) - Start a service, maintenance on a host (DRAIN), a snapshot, or a backup.
* [australis start sla-drain count](australis_start_sla-drain_count.md) - Place a list of space separated Mesos Agents into maintenance mode using the count SLA aware policy as a fallback.
* [australis start sla-drain percentage](australis_start_sla-drain_percentage.md) - Place a list of space separated Mesos Agents into maintenance mode using the percentage SLA aware policy as a fallback.
* [australis start](australis_start.md) - Start a service, maintenance on a host (DRAIN), a snapshot, an update, or a backup.
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -1,45 +0,0 @@
## australis start sla-drain count
Place a list of space separated Mesos Agents into maintenance mode using the count SLA aware policy as a fallback.
### Synopsis
Adds a Mesos Agent to Aurora's Drain list. Tasks will be drained using the count SLA policy as a fallback
when a Job does not have a defined SLA policy.
```
australis start sla-drain count [space separated host list] [flags]
```
### Options
```
--count int Instances count that should be running to meet SLA. (default 5)
--duration RUNNING Minimum time duration a task needs to be RUNNING to be treated as active. (default 45s)
-h, --help help for count
--interval duration Interval at which to poll scheduler. (default 10s)
--sla-limit duration Time limit after which SLA-Aware drain sheds SLA Awareness. (default 1h0m0s)
--timeout duration Time after which the monitor will stop polling and throw an error. (default 20m0s)
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis start sla-drain](australis_start_sla-drain.md) - Place a list of space separated Mesos Agents into maintenance mode using SLA aware strategies.
###### Auto generated by spf13/cobra on 22-Mar-2019

View file

@ -1,45 +0,0 @@
## australis start sla-drain percentage
Place a list of space separated Mesos Agents into maintenance mode using the percentage SLA aware policy as a fallback.
### Synopsis
Adds a Mesos Agent to Aurora's Drain list. Tasks will be drained using the percentage SLA policy as a fallback
when a Job does not have a defined SLA policy.
```
australis start sla-drain percentage [space separated host list] [flags]
```
### Options
```
--duration RUNNING Minimum time duration a task needs to be RUNNING to be treated as active. (default 45s)
-h, --help help for percentage
--interval duration Interval at which to poll scheduler. (default 10s)
--percent float Percentage of instances that should be running to meet SLA. (default 75)
--sla-limit duration Time limit after which SLA-Aware drain sheds SLA Awareness. (default 1h0m0s)
--timeout duration Time after which the monitor will stop polling and throw an error. (default 20m0s)
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis start sla-drain](australis_start_sla-drain.md) - Place a list of space separated Mesos Agents into maintenance mode using SLA aware strategies.
###### Auto generated by spf13/cobra on 22-Mar-2019

View file

@ -0,0 +1,42 @@
## australis start update
Start an update on an Aurora long running service.
### Synopsis
Starts the update process on an Aurora long running service. If no such service exists, the update mechanism
will act as a deployment, creating all new instances based on the requirements in the update configuration.
```
australis start update [update config] [flags]
```
### Options
```
-h, --help help for update
--interval duration Interval at which to poll scheduler. (default 5s)
```
### Options inherited from parent commands
```
-a, --caCertsPath string Path where CA certificates can be found.
-c, --clientCert string Client certificate to use to connect to Aurora.
-k, --clientKey string Client key to use to connect to Aurora.
--config string Config file to use. (default "/etc/aurora/australis.yml")
-l, --logLevel string Set logging level [panic fatal error warning info debug trace]. (default "info")
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
```
### SEE ALSO
* [australis start](australis_start.md) - Start a service, maintenance on a host (DRAIN), a snapshot, an update, or a backup.
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -23,6 +23,7 @@ Stop a service or maintenance on a host (DRAIN).
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -34,4 +35,4 @@ Stop a service or maintenance on a host (DRAIN).
* [australis stop drain](australis_stop_drain.md) - Stop maintenance on a host (move to NONE).
* [australis stop update](australis_stop_update.md) - Stop update
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -15,7 +15,6 @@ australis stop drain [space separated host list] [flags]
```
-h, --help help for drain
--interval duration Interval at which to poll scheduler. (default 5s)
--timeout duration Time after which the monitor will stop polling and throw an error. (default 1m0s)
```
### Options inherited from parent commands
@ -29,6 +28,7 @@ australis stop drain [space separated host list] [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -38,4 +38,4 @@ australis stop drain [space separated host list] [flags]
* [australis stop](australis_stop.md) - Stop a service or maintenance on a host (DRAIN).
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

View file

@ -30,6 +30,7 @@ australis stop update [update ID] [flags]
-p, --password string Password to use for API authentication
-s, --scheduler_addr string Aurora Scheduler's address.
-i, --skipCertVerification Skip CA certificate hostname verification.
-t, --timeout duration Gorealis timeout. (default 20s)
--toJSON Print output in JSON format.
-u, --username string Username to use for API authentication
-z, --zookeeper string Zookeeper node(s) where Aurora stores information. (comma separated list)
@ -39,4 +40,4 @@ australis stop update [update ID] [flags]
* [australis stop](australis_stop.md) - Stop a service or maintenance on a host (DRAIN).
###### Auto generated by spf13/cobra on 22-Mar-2019
###### Auto generated by spf13/cobra on 8-Sep-2022

24
go.mod
View file

@ -1,16 +1,16 @@
module github.com/rdelval/australis
module github.com/aurora-scheduler/australis
go 1.15
require (
github.com/BurntSushi/toml v0.3.1 // indirect
github.com/cpuguy83/go-md2man v1.0.10 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/paypal/gorealis/v2 v2.21.0
github.com/pkg/errors v0.0.0-20171216070316-e881fd58d78e
github.com/sirupsen/logrus v1.2.0
github.com/spf13/cobra v0.0.0-20180115160933-0c34d16c3123
github.com/spf13/pflag v1.0.3
github.com/spf13/viper v1.3.1
gopkg.in/yaml.v2 v2.2.2
github.com/aurora-scheduler/gorealis/v2 v2.29.0
github.com/pkg/errors v0.9.1
github.com/sirupsen/logrus v1.6.0
github.com/spf13/cobra v1.0.0
github.com/spf13/pflag v1.0.5
github.com/spf13/viper v1.6.3
github.com/stretchr/testify v1.5.0
gopkg.in/yaml.v2 v2.2.8
)
go 1.13
replace github.com/apache/thrift v0.13.0 => github.com/ridv/thrift v0.13.2

216
internal/job.go Normal file
View file

@ -0,0 +1,216 @@
/**
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package internal
import (
"errors"
"strings"
realis "github.com/aurora-scheduler/gorealis/v2"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
)
type URI struct {
URI string `yaml:"uri"`
Extract bool `yaml:"extract"`
Cache bool `yaml:"cache"`
}
type Executor struct {
Name string `yaml:"name"`
Data string `yaml:"data"`
}
type ThermosProcess struct {
Name string `yaml:"name"`
Cmd string `yaml:"cmd"`
}
type DockerContainer struct {
Name string `yaml:"name"`
Tag string `yaml:"tag"`
}
type Container struct {
Docker *DockerContainer `yaml:"docker"`
}
type ValueConstraint struct {
Name string `yaml:"name"`
Values []string `yaml:"values"`
Negated bool `yaml:"negated"`
}
type LimitConstraint struct {
Name string `yaml:"name"`
Limit int32 `yaml:"limit"`
}
type Job struct {
Environment string `yaml:"environment"`
Role string `yaml:"role"`
Name string `yaml:"name"`
CPU float64 `yaml:"cpu"`
RAM int64 `yaml:"ram"`
Disk int64 `yaml:"disk"`
Port int64 `yaml:"port"`
GPU int64 `yaml:"gpu"`
Executor Executor `yaml:"executor"`
Instances int32 `yaml:"instances"`
MaxFailures int32 `yaml:"maxFailures"`
URIs []URI `yaml:"uris"`
Metadata map[string]string `yaml:"labels"`
Service bool `yaml:"service"`
Tier string `yaml:"tier,omitempty" default:"preemptible"`
Priority int32 `yaml:"priority"`
Production bool `yaml:"production"`
Thermos []ThermosProcess `yaml:",flow,omitempty"`
Container *Container `yaml:"container,omitempty"`
CronSchedule *string `yaml:"cronSchedule,omitempty"`
CronCollisionPolicy *string `yaml:"cronCollisionPolicy,omitempty"`
ValueConstraints []ValueConstraint `yaml:"valueConstraints,flow,omitempty"`
LimitConstraints []LimitConstraint `yaml:"limitConstraints,flow,omitempty"`
}
func (j *Job) ToRealis() (*realis.AuroraJob, error) {
auroraJob := realis.NewJob().
Environment(j.Environment).
Role(j.Role).
Name(j.Name).
CPU(j.CPU).
RAM(j.RAM).
Disk(j.Disk).
AddPorts(int(j.Port)).
IsService(j.Service).
Tier(j.Tier).
Priority(j.Priority).
Production(j.Production).
InstanceCount(j.Instances).
MaxFailure(j.MaxFailures)
if j.GPU > 0 {
auroraJob.GPU(j.GPU)
}
if j.CronSchedule != nil {
auroraJob.CronSchedule(*j.CronSchedule)
}
if j.CronCollisionPolicy != nil {
// Ignoring error because we have already checked for it in the validate function
policy, _ := aurora.CronCollisionPolicyFromString(*j.CronCollisionPolicy)
auroraJob.CronCollisionPolicy(policy)
}
// Adding URIs.
for _, uri := range j.URIs {
auroraJob.AddURIs(uri.Extract, uri.Cache, uri.URI)
}
// Adding Metadata.
for key, value := range j.Metadata {
auroraJob.AddLabel(key, value)
}
// If thermos jobs processes are provided, use them
if len(j.Thermos) > 0 {
thermosExec := realis.ThermosExecutor{}
for _, process := range j.Thermos {
thermosExec.AddProcess(realis.NewThermosProcess(process.Name, process.Cmd))
}
auroraJob.ThermosExecutor(thermosExec)
} else if j.Executor.Name != "" {
// Non-Thermos executor
if j.Executor.Name == "" {
return nil, errors.New("no executor name provided")
}
auroraJob.ExecutorName(j.Executor.Name)
auroraJob.ExecutorData(j.Executor.Data)
} else if j.Container != nil {
if j.Container.Docker == nil {
return nil, errors.New("no container specified")
}
if j.Container.Docker.Tag != "" && !strings.ContainsRune(j.Container.Docker.Name, ':') {
j.Container.Docker.Name += ":" + j.Container.Docker.Tag
}
auroraJob.Container(realis.NewDockerContainer().Image(j.Container.Docker.Name))
}
// Setting Constraints
for _, valConstraint := range j.ValueConstraints {
auroraJob.AddValueConstraint(valConstraint.Name, valConstraint.Negated, valConstraint.Values...)
}
for _, limit := range j.LimitConstraints {
auroraJob.AddLimitConstraint(limit.Name, limit.Limit)
}
return auroraJob, nil
}
func (j *Job) Validate() error {
if j.Name == "" {
return errors.New("job name not specified")
}
if j.Role == "" {
return errors.New("job role not specified")
}
if j.Environment == "" {
return errors.New("job environment not specified")
}
if j.Instances <= 0 {
return errors.New("number of instances in job cannot be less than or equal to 0")
}
if j.CPU <= 0.0 {
return errors.New("CPU must be greater than 0")
}
if j.RAM <= 0 {
return errors.New("RAM must be greater than 0")
}
if j.Disk <= 0 {
return errors.New("disk must be greater than 0")
}
if len(j.Thermos) == 0 && j.Executor.Name == "" && j.Container == nil {
return errors.New("task does not contain a thermos definition, a custom executor name, or a container to launch")
}
return nil
}
func (j *Job) ValidateCron() error {
if j.CronSchedule == nil {
return errors.New("cron schedule must be set")
}
if j.CronCollisionPolicy != nil {
if _, err := aurora.CronCollisionPolicyFromString(*j.CronCollisionPolicy); err != nil {
return err
}
} else {
killExisting := aurora.CronCollisionPolicy_KILL_EXISTING.String()
j.CronCollisionPolicy = &killExisting
}
return nil
}

128
internal/updateJob.go Normal file
View file

@ -0,0 +1,128 @@
/**
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package internal
import (
"errors"
"fmt"
"time"
realis "github.com/aurora-scheduler/gorealis/v2"
)
type InstanceRange struct {
First int32 `yaml:"first"`
Last int32 `yaml:"last"`
}
type VariableBatchStrategy struct {
GroupSizes []int32 `yaml:"groupSizes"`
AutoPause bool `yaml:"autoPause"`
}
type BatchStrategy struct {
GroupSize int32 `yaml:"groupSize"`
AutoPause bool `yaml:"autoPause"`
}
type QueueStrategy struct {
GroupSize int32 `yaml:"groupSize"`
}
type UpdateStrategy struct {
VariableBatch *VariableBatchStrategy `yaml:"variableBatch"`
Batch *BatchStrategy `yaml:"batch"`
Queue *QueueStrategy `yaml:"queue"`
}
type UpdateSettings struct {
MaxPerInstanceFailures int32 `yaml:"maxPerInstanceFailures"`
MaxFailedInstances int32 `yaml:"maxFailedInstances"`
MinTimeInRunning time.Duration `yaml:"minTimeInRunning"`
RollbackOnFailure bool `yaml:"rollbackOnFailure"`
InstanceRanges []InstanceRange `yaml:"instanceRanges"`
InstanceCount int32 `yaml:"instanceCount"`
PulseTimeout time.Duration `yaml:"pulseTimeout"`
SLAAware bool `yaml:"slaAware"`
Strategy UpdateStrategy `yaml:"strategy"`
}
func (u *UpdateSettings) Validate() error {
if u.InstanceCount <= 0 {
return errors.New("instance count must be larger than 0")
}
if u.Strategy.VariableBatch != nil {
if len(u.Strategy.VariableBatch.GroupSizes) == 0 {
return errors.New("variable batch strategy must specify at least one batch size")
}
for _, batch := range u.Strategy.VariableBatch.GroupSizes {
if batch <= 0 {
return errors.New("all groups in a variable batch strategy must be larger than 0")
}
}
} else if u.Strategy.Batch != nil {
if u.Strategy.Batch.GroupSize <= 0 {
return errors.New("batch strategy must specify a group larger than 0")
}
} else if u.Strategy.Queue != nil {
if u.Strategy.Queue.GroupSize <= 0 {
return errors.New("queue strategy must specify a group larger than 0")
}
} else {
log.Info("No strategy set, falling back on queue strategy with a group size 1")
}
return nil
}
type UpdateJob struct {
JobConfig Job `yaml:"jobConfig"`
UpdateSettings UpdateSettings `yaml:"updateSettings"`
}
func (u *UpdateJob) ToRealis() (*realis.JobUpdate, error) {
jobConfig, err := u.JobConfig.ToRealis()
if err != nil {
return nil, fmt.Errorf("invalid job configuration %w", err)
}
update := realis.JobUpdateFromAuroraTask(jobConfig.AuroraTask())
update.MaxPerInstanceFailures(u.UpdateSettings.MaxPerInstanceFailures).
MaxFailedInstances(u.UpdateSettings.MaxFailedInstances).
WatchTime(u.UpdateSettings.MinTimeInRunning).
RollbackOnFail(u.UpdateSettings.RollbackOnFailure).
PulseIntervalTimeout(u.UpdateSettings.PulseTimeout).
SlaAware(u.UpdateSettings.SLAAware).
InstanceCount(u.UpdateSettings.InstanceCount)
strategy := u.UpdateSettings.Strategy
switch {
case strategy.VariableBatch != nil:
update.VariableBatchStrategy(strategy.VariableBatch.AutoPause, strategy.VariableBatch.GroupSizes...)
case strategy.Batch != nil:
update.BatchUpdateStrategy(strategy.Batch.AutoPause, strategy.Batch.GroupSize)
case strategy.Queue != nil:
update.QueueUpdateStrategy(strategy.Queue.GroupSize)
default:
update.QueueUpdateStrategy(1)
}
for _, r := range u.UpdateSettings.InstanceRanges {
update.AddInstanceRange(r.First, r.Last)
}
return update, nil
}

161
internal/util.go Normal file
View file

@ -0,0 +1,161 @@
/**
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package internal
import (
"bytes"
"encoding/json"
"fmt"
"os"
"time"
"github.com/aurora-scheduler/gorealis/v2/gen-go/apache/aurora"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
yaml "gopkg.in/yaml.v2"
)
type MonitorCmdConfig struct {
Cmd *cobra.Command
MonitorInterval, MonitorTimeout time.Duration
StatusList []string
}
var log *logrus.Logger
// Logger sets the logger available to the internal package
func Logger(l *logrus.Logger) {
log = l
}
// ToJSON converts an interface to a JSON formatted string
func ToJSON(v interface{}) string {
output, err := json.Marshal(v)
if err != nil {
log.Fatalf("Unable to serialize Aurora response: %+v", v)
}
return string(output)
}
func GetLoggingLevels() string {
var buffer bytes.Buffer
for _, level := range logrus.AllLevels {
buffer.WriteString(level.String())
buffer.WriteString(" ")
}
buffer.Truncate(buffer.Len() - 1)
return buffer.String()
}
func MaintenanceMonitorPrint(hostResult map[string]bool, desiredStates []aurora.MaintenanceMode, toJson bool) {
if len(hostResult) > 0 {
// Create anonymous struct for JSON formatting
output := struct {
DesiredStates []string `json:"desired_states"`
Transitioned []string `json:"transitioned"`
NonTransitioned []string `json:"non-transitioned"`
}{
make([]string, 0),
make([]string, 0),
make([]string, 0),
}
for _, state := range desiredStates {
output.DesiredStates = append(output.DesiredStates, state.String())
}
for host, ok := range hostResult {
if ok {
output.Transitioned = append(output.Transitioned, host)
} else {
output.NonTransitioned = append(output.NonTransitioned, host)
}
}
if toJson {
fmt.Println(ToJSON(output))
} else {
fmt.Printf("Entered %v status: %v\n", output.DesiredStates, output.Transitioned)
fmt.Printf("Did not enter %v status: %v\n", output.DesiredStates, output.NonTransitioned)
}
}
}
func UnmarshalJob(filename string) (Job, error) {
job := Job{}
if jobsFile, err := os.Open(filename); err != nil {
return job, errors.Wrap(err, "unable to read the job config file")
} else {
if err := yaml.NewDecoder(jobsFile).Decode(&job); err != nil {
return job, errors.Wrap(err, "unable to parse job config file")
}
if err := job.Validate(); err != nil {
return job, fmt.Errorf("invalid job config %w", err)
}
}
return job, nil
}
func UnmarshalTaskConfig(filename string) (*aurora.TaskConfig, error) {
if jobsFile, err := os.Open(filename); err != nil {
return nil, errors.Wrap(err, "unable to read the task config file")
} else {
job := Job{}
if err := yaml.NewDecoder(jobsFile).Decode(&job); err != nil {
return nil, errors.Wrap(err, "unable to parse task config file")
}
if auroraJob, err := job.ToRealis(); err != nil {
return nil, errors.Wrap(err, "unable to parse task config file")
} else {
return auroraJob.JobConfig().TaskConfig, nil
}
}
return nil, nil
}
func UnmarshalUpdate(filename string) (UpdateJob, error) {
updateJob := UpdateJob{}
if jobsFile, err := os.Open(filename); err != nil {
return updateJob, errors.Wrap(err, "unable to read the job config file")
} else {
if err := yaml.NewDecoder(jobsFile).Decode(&updateJob); err != nil {
return updateJob, errors.Wrap(err, "unable to parse job config file")
}
if err := updateJob.JobConfig.Validate(); err != nil {
return updateJob, fmt.Errorf("invalid job config %w", err)
}
if err := updateJob.UpdateSettings.Validate(); err != nil {
return updateJob, fmt.Errorf("invalid update configuration %w", err)
}
}
return updateJob, nil
}

View file

@ -12,27 +12,31 @@
* limitations under the License.
*/
package realis_test
package internal
import (
"fmt"
"testing"
realis "github.com/paypal/gorealis"
"github.com/stretchr/testify/assert"
)
func TestLoadClusters(t *testing.T) {
clusters, err := realis.LoadClusters("examples/clusters.json")
if err != nil {
fmt.Print(err)
}
assert.Equal(t, clusters["devcluster"].Name, "devcluster")
assert.Equal(t, clusters["devcluster"].ZK, "192.168.33.7")
assert.Equal(t, clusters["devcluster"].SchedZKPath, "/aurora/scheduler")
assert.Equal(t, clusters["devcluster"].AuthMechanism, "UNAUTHENTICATED")
assert.Equal(t, clusters["devcluster"].AgentRunDir, "latest")
assert.Equal(t, clusters["devcluster"].AgentRoot, "/var/lib/mesos")
func TestUnmarshalJob(t *testing.T) {
_, err := UnmarshalJob("../test/hello_world.yaml")
assert.NoError(t, err)
}
func TestUnmarshalDedicatedJob(t *testing.T) {
_, err := UnmarshalJob("../test/hello_world_dedicated.yaml")
assert.NoError(t, err)
}
func TestUnmarshalCron(t *testing.T) {
cron, err := UnmarshalJob("../test/hello_world_cron.yaml")
assert.NoError(t, err)
assert.NoError(t, cron.ValidateCron())
}
func TestUnmarshalUpdate(t *testing.T) {
_, err := UnmarshalUpdate("../test/update_hello_world.yaml")
assert.NoError(t, err)
}

View file

@ -15,7 +15,7 @@
package main
import (
"github.com/rdelval/australis/cmd"
"github.com/aurora-scheduler/australis/cmd"
)
func main() {

26
test/hello_world.yaml Normal file
View file

@ -0,0 +1,26 @@
---
environment: "prod"
role: "vagrant"
name: "hello_world"
cpu: 0.09
ram: 64
disk: 128
instances: 1
thermos:
- name: "bootstrap"
cmd: "echo bootstrapping"
- name: "hello_gorealis"
cmd: "while true; do echo hello world from gorealis; sleep 10; done"
updateSettings:
maxPerInstanceFailures: 1
maxFailedInstances: 1
minTimeInRunning: 1m
rollbackOnFailure: true
instanceRanges:
- start: 1
end: 4
blockIfNoPulseAfter: 1m
slaAware: false
strategy:
name: Batch
groupSize: 2

View file

@ -6,8 +6,10 @@ cpu: 0.09
ram: 64
disk: 128
instances: 1
cronSchedule: "*/1 * * * *"
cronCollisionPolicy: "CANCEL_NEW"
thermos:
- name: "bootstrap"
cmd: "echo bootstrapping"
- name: "hello_gorealis"
cmd: "while true; do echo hello world from gorealis; sleep 10; done"
cmd: "echo hello world from gorealis; sleep 10;"

View file

@ -0,0 +1,30 @@
---
environment: "prod"
role: "vagrant"
name: "hello_world"
cpu: 0.09
ram: 64
disk: 128
instances: 1
valueConstraints:
- name: "dedicated"
values:
- "vagrant/bar"
thermos:
- name: "bootstrap"
cmd: "echo bootstrapping"
- name: "hello_gorealis"
cmd: "while true; do echo hello world from gorealis; sleep 10; done"
updateSettings:
maxPerInstanceFailures: 1
maxFailedInstances: 1
minTimeInRunning: 1m
rollbackOnFailure: true
instanceRanges:
- start: 1
end: 4
blockIfNoPulseAfter: 1m
slaAware: false
strategy:
name: Batch
groupSize: 2

20
test/task_config.yaml Normal file
View file

@ -0,0 +1,20 @@
environment: "prod"
role: "vagrant"
name: "hello_world"
cpu: 0.09
ram: 64
disk: 128
valueConstraints:
- name: "dedicated"
values:
- "vagrant/bar"
limitConstraints:
- name: "host"
limit: 1
- name: "zone"
limit: 2
thermos:
- name: "bootstrap"
cmd: "echo bootstrapping"
- name: "hello_gorealis"
cmd: "while true; do echo hello world from gorealis; sleep 10; done"

View file

@ -0,0 +1,29 @@
---
jobConfig:
environment: "prod"
role: "vagrant"
name: "hello_world"
cpu: 0.09
ram: 64
disk: 128
instances: 1
maxFailures: 1
thermos:
- name: "bootstrap"
cmd: "echo bootstrapping"
- name: "hello_gorealis"
cmd: "while true; do echo hello world from gorealis; sleep 10; done"
updateSettings:
maxPerInstanceFailures: 1
maxFailedInstances: 1
minTimeInRunning: 1m
rollbackOnFailure: true
instanceCount: 1
instanceRanges:
- first: 1
last: 4
pulseTimeout: 1m
strategy:
variableBatch:
groupSizes: [1,2,3]
autoPause: true

View file

@ -1,56 +0,0 @@
---
Language: Cpp
# BasedOnStyle: LLVM
AccessModifierOffset: -2
ConstructorInitializerIndentWidth: 2
AlignEscapedNewlinesLeft: false
AlignTrailingComments: true
AllowAllParametersOfDeclarationOnNextLine: false
AllowShortBlocksOnASingleLine: false
AllowShortIfStatementsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AllowShortFunctionsOnASingleLine: Inline
AlwaysBreakTemplateDeclarations: true
AlwaysBreakBeforeMultilineStrings: true
BreakBeforeBinaryOperators: true
BreakBeforeTernaryOperators: true
BreakConstructorInitializersBeforeComma: false
BinPackParameters: false
ColumnLimit: 100
ConstructorInitializerAllOnOneLineOrOnePerLine: true
DerivePointerAlignment: false
IndentCaseLabels: false
IndentWrappedFunctionNames: false
IndentFunctionDeclarationAfterType: false
MaxEmptyLinesToKeep: 1
KeepEmptyLinesAtTheStartOfBlocks: true
NamespaceIndentation: None
ObjCSpaceAfterProperty: false
ObjCSpaceBeforeProtocolList: true
PenaltyBreakBeforeFirstCallParameter: 190
PenaltyBreakComment: 300
PenaltyBreakString: 10000
PenaltyBreakFirstLessLess: 120
PenaltyExcessCharacter: 1000000
PenaltyReturnTypeOnItsOwnLine: 1200
PointerAlignment: Left
SpacesBeforeTrailingComments: 1
Cpp11BracedListStyle: true
Standard: Auto
IndentWidth: 2
TabWidth: 4
UseTab: Never
BreakBeforeBraces: Attach
SpacesInParentheses: false
SpacesInAngles: false
SpaceInEmptyParentheses: false
SpacesInCStyleCastParentheses: false
SpacesInContainerLiterals: true
SpaceBeforeAssignmentOperators: true
ContinuationIndentWidth: 4
CommentPragmas: '^ IWYU pragma:'
ForEachMacros: [ foreach, Q_FOREACH, BOOST_FOREACH ]
SpaceBeforeParens: ControlStatements
DisableFormat: false
...

View file

@ -1 +0,0 @@
.git/

View file

@ -1,112 +0,0 @@
#
## Licensed to the Apache Software Foundation (ASF) under one
## or more contributor license agreements. See the NOTICE file
## distributed with this work for additional information
## regarding copyright ownership. The ASF licenses this file
## to you under the Apache License, Version 2.0 (the
## "License"); you may not use this file except in compliance
## with the License. You may obtain a copy of the License at
##
## http://www.apache.org/licenses/LICENSE-2.0
##
## Unless required by applicable law or agreed to in writing,
## software distributed under the License is distributed on an
## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
## KIND, either express or implied. See the License for the
## specific language governing permissions and limitations
## under the License.
##
#
# EditorConfig: http://editorconfig.org
# see doc/coding_standards.md
root = true
[*]
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
# ActionScript
# [*.as]
# C
# [*.c]
# C++
[*.cpp]
indent_style = space
indent_size = 2
# C-Sharp
# [*.cs]
# D
# [*.d]
# Erlang
# [*.erl]
# Go-lang
[*.go]
indent_style = tab
indent_size = 8
# C header files
# [*.h]
# Haskell
# [*.hs]
# Haxe
# [*.hx]
# Java
# [*.java]
# Javascript
[*.js]
indent_style = space
indent_size = 2
# JSON
[*.json]
indent_style = space
indent_size = 2
# Lua
# [*.lua]
[*.markdown]
indent_style = space
trim_trailing_whitespace = false
[*.md]
indent_style = space
trim_trailing_whitespace = false
# OCaml
# [*.ml]
# Delphi Pascal
# [*.pas]
# PHP
# [*.php]
# Perl
# [*.pm]
# Python
# [*.py]
# Ruby
# [*.rb]
# Typescript
# [*.ts]
# XML
# [*.xml]

View file

@ -1,9 +0,0 @@
# TODO: Use eslint on js lib and generated code
# Ignore lib/js for now, which uses jshint currently
lib/js/*
# Ignore all generated code for now
**/gen-*
# Don't lint nested node_modules
**/node_modules

View file

@ -1,24 +0,0 @@
{
"env": {
"es6": true,
"node": true
},
"extends": [
"eslint:recommended",
"plugin:prettier/recommended"
],
"parserOptions": {
"ecmaVersion": 2017
},
"rules": {
"no-console": "off",
"no-var": "error",
"prefer-const": "error",
"no-constant-condition": [
"error",
{
"checkLoops": false
}
]
}
}

View file

@ -1 +0,0 @@
* text=auto

View file

@ -1,59 +0,0 @@
# Configuration for probot-stale - https://github.com/probot/stale
# Number of days of inactivity before an Issue or Pull Request becomes stale
daysUntilStale: 60
# Number of days of inactivity before an Issue or Pull Request with the stale label is closed.
# Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale.
daysUntilClose: 7
# Issues or Pull Requests with these labels will never be considered stale. Set to `[]` to disable
exemptLabels:
- Do Not Merge
- blocked
- pinned
- security
- "[Status] Maybe Later"
# Set to true to ignore issues in a project (defaults to false)
exemptProjects: false
# Set to true to ignore issues in a milestone (defaults to false)
exemptMilestones: false
# Label to use when marking as stale
staleLabel: wontfix
# Comment to post when marking as stale. Set to `false` to disable
markComment: >
This issue has been automatically marked as stale because it has not had
recent activity. It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.
# Comment to post when removing the stale label.
unmarkComment: >
This issue is no longer stale.
Thank you for your contributions.
# Comment to post when closing a stale Issue or Pull Request.
closeComment: >
This issue has been automatically closed due to inactivity.
Thank you for your contributions.
# Limit the number of actions per hour, from 1-30. Default is 30
limitPerRun: 30
# Limit to only `issues` or `pulls`
# only: issues
# Optionally, specify configuration settings that are specific to just 'issues' or 'pulls':
# pulls:
# daysUntilStale: 30
# markComment: >
# This pull request has been automatically marked as stale because it has not had
# recent activity. It will be closed if no further activity occurs. Thank you
# for your contributions.
# issues:
# exemptLabels:
# - confirmed

View file

@ -1,395 +0,0 @@
# generic ignores
*.la
*.lo
*.o
*.deps
*.dirstamp
*.libs
*.log
*.trs
*.suo
*.pyc
*.cache
*.user
*.ipch
*.sdf
*.jar
*.exe
*.dll
*_ReSharper*
*.opensdf
*.swp
*.hi
*~
tags
.*project
.classpath
.dub
.settings
.checkstyle
junit*.properties
.idea
*.iml
*.ipr
*.iws
gen-*
Makefile
Makefile.in
aclocal.m4
acinclude.m4
apache-thrift-test-library
autom4te.cache
cmake-*
dub.selections.json
libapache-thrift.a
node_modules
compile
test-driver
erl_crash.dump
project.lock.json
.sonar
.DS_Store
.svn
.vagrant
.vscode
.vs
/aclocal/libtool.m4
/aclocal/lt*.m4
/autoscan.log
/autoscan-*.log
/cmake_*
/compiler/cpp/compiler.VC.db
/compiler/cpp/compiler.VC.VC.opendb
/compiler/cpp/test/plugin/t_cpp_generator.cc
/compiler/cpp/src/thrift/plugin/plugin_constants.cpp
/compiler/cpp/src/thrift/plugin/plugin_constants.h
/compiler/cpp/src/thrift/plugin/plugin_types.cpp
/compiler/cpp/src/thrift/plugin/plugin_types.h
/compiler/cpp/test/*test
/compiler/cpp/test/thrift-gen-*
/compiler/cpp/src/thrift/thrift-bootstrap
/compiler/cpp/src/thrift/plugin/gen.stamp
/compiler/cpp/Debug
/compiler/cpp/Release
/compiler/cpp/src/thrift/libparse.a
/compiler/cpp/src/thrift/thriftl.cc
/compiler/cpp/src/thrift/thrifty.cc
/compiler/cpp/src/thrift/thrifty.hh
/compiler/cpp/src/thrift/windows/version.h
/compiler/cpp/thrift
/compiler/cpp/thriftl.cc
/compiler/cpp/thrifty.cc
/compiler/cpp/lex.yythriftl.cc
/compiler/cpp/thrifty.h
/compiler/cpp/thrifty.hh
/compiler/cpp/src/thrift/version.h
/config.*
/configure
/configure.lineno
/configure.scan
/contrib/.vagrant/
/contrib/fb303/config.cache
/contrib/fb303/config.log
/contrib/fb303/config.status
/contrib/fb303/configure
/contrib/fb303/cpp/libfb303.a
/contrib/fb303/java/build/
/contrib/fb303/py/build/
/contrib/fb303/py/fb303/FacebookService-remote
/contrib/fb303/py/fb303/FacebookService.py
/contrib/fb303/py/fb303/__init__.py
/contrib/fb303/py/fb303/constants.py
/contrib/fb303/py/fb303/ttypes.py
/depcomp
/install-sh
/lib/cl/backport-update.zip
/lib/cl/lib
/lib/cl/run-tests
/lib/cl/quicklisp.lisp
/lib/cl/externals/
/lib/cl/run-tests
/lib/cl/quicklisp/
/lib/cpp/Debug/
/lib/cpp/Debug-mt/
/lib/cpp/Release/
/lib/cpp/Release-mt/
/lib/cpp/src/thrift/qt/moc_TQTcpServer.cpp
/lib/cpp/src/thrift/qt/moc__TQTcpServer.cpp
/lib/cpp/src/thrift/config.h
/lib/cpp/src/thrift/stamp-h2
/lib/cpp/test/Benchmark
/lib/cpp/test/AllProtocolsTest
/lib/cpp/test/AnnotationTest
/lib/cpp/test/DebugProtoTest
/lib/cpp/test/DenseProtoTest
/lib/cpp/test/EnumTest
/lib/cpp/test/JSONProtoTest
/lib/cpp/test/OptionalRequiredTest
/lib/cpp/test/SecurityTest
/lib/cpp/test/SpecializationTest
/lib/cpp/test/RecursiveTest
/lib/cpp/test/ReflectionTest
/lib/cpp/test/RenderedDoubleConstantsTest
/lib/cpp/test/TFDTransportTest
/lib/cpp/test/TFileTransportTest
/lib/cpp/test/TInterruptTest
/lib/cpp/test/TNonblockingServerTest
/lib/cpp/test/TNonblockingSSLServerTest
/lib/cpp/test/TPipedTransportTest
/lib/cpp/test/TServerIntegrationTest
/lib/cpp/test/TSocketInterruptTest
/lib/cpp/test/TransportTest
/lib/cpp/test/UnitTests
/lib/cpp/test/ZlibTest
/lib/cpp/test/OpenSSLManualInitTest
/lib/cpp/test/concurrency_test
/lib/cpp/test/link_test
/lib/cpp/test/processor_test
/lib/cpp/test/tests.xml
/lib/cpp/concurrency_test
/lib/cpp/*.pc
/lib/cpp/x64/Debug/
/lib/cpp/x64/Debug-mt/
/lib/cpp/x64/Release
/lib/cpp/x64/Release-mt
/lib/c_glib/*.gcda
/lib/c_glib/*.gcno
/lib/c_glib/*.loT
/lib/c_glib/src/thrift/config.h
/lib/c_glib/src/thrift/stamp-h3
/lib/c_glib/test/*.gcno
/lib/c_glib/test/testwrapper.sh
/lib/c_glib/test/testwrapper-test*
/lib/c_glib/test/testapplicationexception
/lib/c_glib/test/testbinaryprotocol
/lib/c_glib/test/testcompactprotocol
/lib/c_glib/test/testbufferedtransport
/lib/c_glib/test/testcontainertest
/lib/c_glib/test/testdebugproto
/lib/c_glib/test/testfdtransport
/lib/c_glib/test/testframedtransport
/lib/c_glib/test/testmemorybuffer
/lib/c_glib/test/testoptionalrequired
/lib/c_glib/test/testtransportsslsocket
/lib/c_glib/test/testsimpleserver
/lib/c_glib/test/teststruct
/lib/c_glib/test/testthrifttest
/lib/c_glib/test/testthrifttestclient
/lib/c_glib/test/testtransportsocket
/lib/c_glib/test/testserialization
/lib/c_glib/thriftc.pc
/lib/c_glib/thrift_c_glib.pc
/lib/csharp/**/bin/
/lib/csharp/**/obj/
/lib/csharp/src/packages
/lib/d/test/*.pem
/lib/d/libthriftd*.a
/lib/d/test/async_test
/lib/d/test/client_pool_test
/lib/d/test/serialization_benchmark
/lib/d/test/stress_test_server
/lib/d/test/thrift_test_client
/lib/d/test/thrift_test_server
/lib/d/test/transport_test
/lib/d/unittest/
/lib/dart/coverage
/lib/dart/**/.packages
/lib/dart/**/packages
/lib/dart/**/.pub/
/lib/dart/**/pubspec.lock
/lib/delphi/test/skip/*.request
/lib/delphi/test/skip/*.response
/lib/delphi/**/*.identcache
/lib/delphi/**/*.local
/lib/delphi/**/*.dcu
/lib/delphi/**/*.2007
/lib/delphi/**/*.dproj
/lib/delphi/**/codegen/*.bat
/lib/erl/.eunit
/lib/erl/.generated
/lib/erl/.rebar/
/lib/erl/deps/
/lib/erl/ebin
/lib/erl/src/thrift.app.src
/lib/erl/test/*.beam
/lib/erl/test/*.hrl
/lib/erl/test/Thrift_omit_without.thrift
/lib/haxe/test/bin
/lib/haxe/test/data.tmp
/lib/hs/dist
/lib/java/.gradle
/lib/java/android/.gradle
/lib/java/build
/lib/java/target
/lib/js/dist
/lib/js/doc
/lib/js/test/build
/lib/netcore/**/bin
/lib/netcore/**/obj
/lib/nodejs/coverage
/lib/nodejs/node_modules/
/lib/perl/MANIFEST
/lib/perl/MYMETA.json
/lib/perl/MYMETA.yml
/lib/perl/Makefile-perl.mk
/lib/perl/blib
/lib/perl/pm_to_blib
/lib/py/build
/lib/py/thrift.egg-info/
/lib/rb/Gemfile.lock
/lib/rb/debug_proto_test
/lib/rb/.config
/lib/rb/ext/conftest.dSYM/
/lib/rb/ext/mkmf.log
/lib/rb/ext/thrift_native.bundle
/lib/rb/ext/thrift_native.so
/lib/rb/test/
/lib/rb/thrift-*.gem
/lib/php/src/ext/thrift_protocol/Makefile.*
/lib/php/src/ext/thrift_protocol/build/
/lib/php/src/ext/thrift_protocol/config.*
/lib/php/src/ext/thrift_protocol/configure
/lib/php/src/ext/thrift_protocol/configure.ac
/lib/php/src/ext/thrift_protocol/configure.in
/lib/php/src/ext/thrift_protocol/install-sh
/lib/php/src/ext/thrift_protocol/libtool
/lib/php/src/ext/thrift_protocol/ltmain.sh
/lib/php/src/ext/thrift_protocol/missing
/lib/php/src/ext/thrift_protocol/mkinstalldirs
/lib/php/src/ext/thrift_protocol/modules/
/lib/php/src/ext/thrift_protocol/php_thrift_protocol.lo
/lib/php/src/ext/thrift_protocol/run-tests.php
/lib/php/src/ext/thrift_protocol/thrift_protocol.la
/lib/php/src/ext/thrift_protocol/tmp-php.ini
/lib/php/src/packages/
/lib/php/test/TEST-*.xml
/lib/php/test/packages/
/lib/py/dist/
/lib/erl/logs/
/lib/go/pkg
/lib/go/src
/lib/go/test/gopath/
/lib/go/test/ThriftTest.thrift
/lib/rs/target/
/lib/rs/Cargo.lock
/lib/rs/test/Cargo.lock
/lib/rs/test/target/
/lib/rs/test/bin/
/lib/rs/test/src/base_one.rs
/lib/rs/test/src/base_two.rs
/lib/rs/test/src/midlayer.rs
/lib/rs/test/src/recursive.rs
/lib/rs/test/src/ultimate.rs
/lib/rs/*.iml
/lib/rs/**/*.iml
/libtool
/ltmain.sh
/missing
/node_modules/
/vendor/
/composer.lock
/stamp-h1
/test/features/results.json
/test/results.json
/test/c_glib/test_client
/test/c_glib/test_server
/test/cl/TestServer
/test/cl/TestClient
/test/cpp/StressTest
/test/cpp/StressTestNonBlocking
/test/cpp/TestClient
/test/cpp/TestServer
/test/csharp/obj
/test/csharp/bin
/test/dart/**/.packages
/test/dart/**/packages
/test/dart/**/.pub/
/test/dart/**/pubspec.lock
/test/log/
/test/test.log
/test/erl/.generated
/test/erl/.rebar
/test/erl/ebin
/test/go/bin/
/test/go/ThriftTest.thrift
/test/go/gopath
/test/go/pkg/
/test/go/src/code.google.com/
/test/go/src/common/mock_handler.go
/test/go/src/github.com/golang/
/test/go/src/golang.org/
/test/go/src/gen/
/test/go/src/thrift
/test/haxe/bin
/test/hs/TestClient
/test/hs/TestServer
/test/php/php_ext_dir/
/test/py.twisted/_trial_temp/
/test/rb/Gemfile.lock
/test/netcore/**/bin
/test/netcore/**/obj
/test/netcore/Thrift
/test/php/php_ext_dir/
/test/rs/Cargo.lock
/test/rs/src/thrift_test.rs
/test/rs/bin/
/test/rs/target/
/test/rs/*.iml
/test/rs/**/*.iml
/lib/cl/backport-update.zip
/lib/cl/lib
/tutorial/cl/quicklisp.lisp
/tutorial/cl/externals/
/tutorial/cl/quicklisp/
/tutorial/cl/TutorialClient
/tutorial/cl/TutorialServer
/tutorial/cl/backport-update.zip
/tutorial/cl/lib/
/tutorial/cl/shared-implementation.fasl
/tutorial/cl/tutorial-implementation.fasl
/tutorial/cpp/TutorialClient
/tutorial/cpp/TutorialServer
/tutorial/c_glib/tutorial_client
/tutorial/c_glib/tutorial_server
/tutorial/csharp/CsharpServer/obj
/tutorial/csharp/CsharpServer/bin
/tutorial/csharp/CsharpClient/obj
/tutorial/csharp/CsharpClient/bin
/tutorial/d/async_client
/tutorial/d/client
/tutorial/d/server
/tutorial/dart/**/.packages
/tutorial/dart/**/packages
/tutorial/dart/**/.pub/
/tutorial/dart/**/pubspec.lock
/tutorial/delphi/**/*.dsk
/tutorial/delphi/**/*.local
/tutorial/delphi/**/*.tvsconfig
/tutorial/delphi/**/dcu
/tutorial/delphi/**/*.local
/tutorial/delphi/**/*.identcache
/tutorial/go/gopath
/tutorial/go/go-tutorial
/tutorial/go/calculator-remote
/tutorial/go/src/shared
/tutorial/go/src/tutorial
/tutorial/go/src/git.apache.org
/tutorial/go/src/golang.org
/tutorial/haxe/bin
/tutorial/hs/dist/
/tutorial/java/build/
/tutorial/js/build/
/tutorial/netcore/**/bin
/tutorial/netcore/**/obj
/tutorial/netcore/Thrift
/tutorial/rs/*.iml
/tutorial/rs/src/shared.rs
/tutorial/rs/src/tutorial.rs
/tutorial/rs/bin
/tutorial/rs/target
/tutorial/rs/Cargo.lock
/ylwrap

View file

@ -1,64 +0,0 @@
max_width = 100
hard_tabs = false
tab_spaces = 4
newline_style = "Auto"
use_small_heuristics = "Default"
indent_style = "Block"
wrap_comments = false
format_doc_comments = false
comment_width = 80
normalize_comments = false
normalize_doc_attributes = false
license_template_path = ""
format_strings = false
format_macro_matchers = false
format_macro_bodies = true
empty_item_single_line = true
struct_lit_single_line = true
fn_single_line = false
where_single_line = false
imports_indent = "Block"
imports_layout = "Mixed"
merge_imports = false
reorder_imports = true
reorder_modules = true
reorder_impl_items = false
type_punctuation_density = "Wide"
space_before_colon = false
space_after_colon = true
spaces_around_ranges = false
binop_separator = "Front"
remove_nested_parens = true
combine_control_expr = true
overflow_delimited_expr = false
struct_field_align_threshold = 0
enum_discrim_align_threshold = 0
match_arm_blocks = true
force_multiline_blocks = false
fn_args_density = "Tall"
brace_style = "SameLineWhere"
control_brace_style = "AlwaysSameLine"
trailing_semicolon = true
trailing_comma = "Vertical"
match_block_trailing_comma = false
blank_lines_upper_bound = 1
blank_lines_lower_bound = 0
edition = "2015"
merge_derives = true
use_try_shorthand = false
use_field_init_shorthand = false
force_explicit_abi = true
condense_wildcard_suffixes = false
color = "Auto"
required_version = "1.0.0"
unstable_features = false
disable_all_formatting = false
skip_children = false
hide_parse_errors = false
error_on_line_overflow = false
error_on_unformatted = false
report_todo = "Never"
report_fixme = "Never"
ignore = []
emit_mode = "Files"
make_backup = false

View file

@ -1,181 +0,0 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
# build Apache Thrift on Travis CI - https://travis-ci.org/
#
# Docker Integration
# see: build/docker/README.md
#
sudo: required
dist: trusty
language: cpp
services:
- docker
install:
- if [[ `uname` == "Linux" ]]; then build/docker/refresh.sh; fi
stages:
- docker # docker images
- thrift # thrift build jobs
env:
global:
- SCRIPT="cmake.sh"
- BUILD_ARG=""
- BUILD_ENV="-e CC=gcc -e CXX=g++ -e THRIFT_CROSSTEST_CONCURRENCY=4"
- DISTRO=ubuntu-bionic
- BUILD_LIBS="CPP C_GLIB HASKELL JAVA PYTHON TESTING TUTORIALS" # only meaningful for CMake builds
- TRAVIS_BUILD_STAGE=test
# DOCKER_REPO (this works for all builds as a source for docker images - you can override for fork builds in your Travis settings)
- DOCKER_REPO="thrift/thrift-build"
# DOCKER_USER (provide in your Travis settings if you want to build and update docker images once, instead of on every job)
# DOCKER_PASS (same)
jobs:
include:
# ========================= stage: docker =========================
- stage: docker
script: true
env:
- JOB="Docker Build ubuntu-xenial 16.04 LTS"
- DISTRO=ubuntu-xenial
- TRAVIS_BUILD_STAGE=docker
- script: true
env:
- JOB="Docker Build ubuntu-bionic 18.04 LTS"
- DISTRO=ubuntu-bionic
- TRAVIS_BUILD_STAGE=docker
# ========================= stage: thrift =======================
# ------------------------- phase: cross ------------------------
# apache/thrift official PR builds can exceed 50 minutes per job so combine all cross tests
- stage: thrift
script: build/docker/run.sh
if: repo = apache/thrift
env:
- JOB="Cross Language Tests"
- SCRIPT="cross-test.sh"
# fork based PR builds cannot exceed 50 minutes per job
- stage: thrift
script: build/docker/run.sh
if: repo != apache/thrift
env:
- JOB="Cross Language Tests (Binary Protocol)"
- SCRIPT="cross-test.sh"
- BUILD_ARG="-'(binary)'"
- stage: thrift
script: build/docker/run.sh
if: repo != apache/thrift
env:
- JOB="Cross Language Tests (Header, JSON Protocols)"
- SCRIPT="cross-test.sh"
- BUILD_ARG="-'(header|json)'"
- stage: thrift
script: build/docker/run.sh
if: repo != apache/thrift
env:
- JOB="Cross Language Tests (Compact and Multiplexed Protocols)"
- SCRIPT="cross-test.sh"
- BUILD_ARG="-'(compact|multiplexed)'"
# ------------------------- phase: sca --------------------------
# QA jobs for code analytics and metrics
- stage: thrift
script: build/docker/run.sh
env:
- JOB="Static Code Analysis"
- SCRIPT="sca.sh"
# C and C++ undefined behavior.
# A binary crashes if undefined behavior occurs and produces a stack trace.
# python is disabled, see: THRIFT-4360
- script: build/docker/run.sh
env:
- JOB="UBSan"
- SCRIPT="ubsan.sh"
- BUILD_ARG="--without-python --without-py3"
# ------------------------- phase: autotools --------------------
# TODO: Remove them once migrated to CMake
- script: build/docker/run.sh
env:
- JOB="Autotools (Ubuntu Bionic)"
- SCRIPT="autotools.sh"
- script: build/docker/run.sh
env:
- JOB="Autotools (Ubuntu Xenial)"
- DISTRO=ubuntu-xenial
- SCRIPT="autotools.sh"
# ------------------------- phase: cmake ------------------------
- script: build/docker/run.sh
env:
- JOB="CMake"
# C++ specific options: compiler plug-in, threading model
- script: build/docker/run.sh
env:
- JOB="C++98 (Boost Thread)"
- SCRIPT="cmake.sh"
- BUILD_LIBS="CPP TESTING TUTORIALS"
- BUILD_ARG="-DCMAKE_CXX_STANDARD=98 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DCMAKE_CXX_EXTENSIONS=OFF --DWITH_BOOSTTHREADS=ON -DWITH_PYTHON=OFF -DWITH_C_GLIB=OFF -DWITH_JAVA=OFF -DWITH_HASKELL=OFF"
- BUILD_ENV="-e CC=clang -e CXX=clang++"
- script: build/docker/run.sh
env:
- JOB="C++ (Std Thread) and Plugin"
- SCRIPT="cmake.sh"
- BUILD_LIBS="CPP TESTING TUTORIALS"
- BUILD_ARG="-DWITH_PLUGIN=ON -DWITH_STDTHREADS=ON -DWITH_PYTHON=OFF -DWITH_C_GLIB=OFF -DWITH_JAVA=OFF -DWITH_HASKELL=OFF"
- BUILD_ENV="-e CC=clang -e CXX=clang++"
# ------------------------- phase: dist -------------------------
- script: build/docker/run.sh
env:
- JOB="make dist"
- SCRIPT="make-dist.sh"
- script: build/docker/run.sh
env:
- JOB="Debian Packages"
- SCRIPT="dpkg.sh"
# ------------------------- phase: coverity ---------------------
# We build the coverity scan build once monthly using a travis cron job
- if: (env(COVERITY_SCAN_NOTIFICATION_EMAIL) IS present) AND (branch IN (master)) AND (type IN (cron))
script: build/docker/run.sh
env:
- JOB="Coverity Scan"
- SCRIPT="covscan.sh"
### ------------------------- phase: osx --------------------------
# disabled due to the time delays it imposes on build jobs
# - os: osx
# osx_image: xcode9
# script: build/docker/scripts/autotools.sh

File diff suppressed because it is too large Load diff

View file

@ -1,124 +0,0 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
cmake_minimum_required(VERSION 3.1)
# CMake 3.1 supports C++ standards selection with CMAKE_CXX_STANDARD
# If you need CMake 3.1+ for Ubuntu 14.04, try
# https://launchpad.net/~george-edison55/+archive/ubuntu/cmake-3.x
# If you need CMake 3.1+ for debian "jessie", get it from jessie-backports
# Otherwise
# http://cmake.org
project("Apache Thrift")
set(CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH}" "${CMAKE_CURRENT_SOURCE_DIR}/build/cmake")
# TODO: add `git rev-parse --short HEAD`
# Read the version information from the Autoconf file
file (STRINGS "${CMAKE_CURRENT_SOURCE_DIR}/configure.ac" CONFIGURE_AC REGEX "AC_INIT\\(.*\\)" )
# The following variable is used in the version.h.in file
string(REGEX REPLACE "AC_INIT\\(\\[.*\\], \\[([0-9]+\\.[0-9]+\\.[0-9]+(-dev)?)\\]\\)" "\\1" PACKAGE_VERSION ${CONFIGURE_AC})
message(STATUS "Parsed Thrift package version: ${PACKAGE_VERSION}")
# These are internal to CMake
string(REGEX REPLACE "([0-9]+\\.[0-9]+\\.[0-9]+)(-dev)?" "\\1" thrift_VERSION ${PACKAGE_VERSION})
string(REGEX REPLACE "([0-9]+)\\.[0-9]+\\.[0-9]+" "\\1" thrift_VERSION_MAJOR ${thrift_VERSION})
string(REGEX REPLACE "[0-9]+\\.([0-9])+\\.[0-9]+" "\\1" thrift_VERSION_MINOR ${thrift_VERSION})
string(REGEX REPLACE "[0-9]+\\.[0-9]+\\.([0-9]+)" "\\1" thrift_VERSION_PATCH ${thrift_VERSION})
message(STATUS "Parsed Thrift version: ${thrift_VERSION} (${thrift_VERSION_MAJOR}.${thrift_VERSION_MINOR}.${thrift_VERSION_PATCH})")
# Some default settings
include(DefineCMakeDefaults)
# Build time options are defined here
include(DefineOptions)
include(DefineInstallationPaths)
# Based on the options set some platform specifics
include(DefinePlatformSpecifc)
# Generate the config.h file
include(ConfigureChecks)
# Package it
include(CPackConfig)
find_package(Threads)
include(CTest)
if(BUILD_TESTING)
message(STATUS "Building with unittests")
enable_testing()
# Define "make check" as alias for "make test"
add_custom_target(check COMMAND ctest)
else ()
message(STATUS "Building without tests")
endif ()
if(BUILD_COMPILER)
if(NOT EXISTS ${THRIFT_COMPILER})
set(THRIFT_COMPILER $<TARGET_FILE:thrift-compiler>)
endif()
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/compiler/cpp)
elseif(EXISTS ${THRIFT_COMPILER})
add_executable(thrift-compiler IMPORTED)
set_property(TARGET thrift-compiler PROPERTY IMPORTED_LOCATION ${THRIFT_COMPILER})
endif()
if(BUILD_CPP)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/lib/cpp)
if(BUILD_TUTORIALS)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/tutorial/cpp)
endif()
if(BUILD_TESTING)
if(WITH_LIBEVENT AND WITH_ZLIB AND WITH_OPENSSL)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/test/cpp)
else()
message(WARNING "libevent and/or ZLIB and/or OpenSSL not found or disabled; will not build some tests")
endif()
endif()
endif()
if(BUILD_C_GLIB)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/lib/c_glib)
endif()
if(BUILD_JAVA)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/lib/java)
endif()
if(BUILD_PYTHON)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/lib/py)
if(BUILD_TESTING)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/test/py)
endif()
endif()
if(BUILD_HASKELL)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/lib/hs)
if(BUILD_TESTING)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/test/hs)
endif()
endif()
PRINT_CONFIG_SUMMARY()

View file

@ -1,116 +0,0 @@
# How to Contribute #
Thank you for your interest in contributing to the Apache Thrift project! Information on why and how to contribute is available on the Apache Software Foundation (ASF) web site. In particular, we recommend the following to become acquainted with Apache Contributions:
* [Contributors Tech Guide](http://www.apache.org/dev/contributors)
* [Get involved!](http://www.apache.org/foundation/getinvolved.html)
* [Legal aspects on Submission of Contributions (Patches)](http://www.apache.org/licenses/LICENSE-2.0.html#contributions)
## GitHub pull requests ##
This is the preferred method of submitting changes. When you submit a pull request through github,
it activates the continuous integration (CI) build systems at Appveyor and Travis to build your changesxi
on a variety of Linux and Windows configurations and run all the test suites. Follow these requirements
for a successful pull request:
1. All code changes require an [Apache Jira THRIFT Issue](http://issues.apache.org/jira/browse/THRIFT) ticket.
1. All pull requests should contain a single commit per issue, or we will ask you to squash it.
1. The pull request title must begin with the Jira THRIFT ticket identifier, for example:
THRIFT-9999: an example pull request title
1. Commit messages must follow this pattern for code changes (deviations will not be merged):
THRIFT-9999: [summary of fix, one line if possible]
Client: [language(s) affected, comma separated, use lib/ directory names please]
Instructions:
1. Create a fork in your GitHub account of http://github.com/apache/thrift
1. Clone the fork to your development system.
1. Create a branch for your changes (best practice is issue as branch name, e.g. THRIFT-9999).
1. Modify the source to include the improvement/bugfix, and:
* Remember to provide *tests* for all submitted changes!
* Use test-driven development (TDD): add a test that will isolate the bug *before* applying the change that fixes it.
* Verify that you follow [Thrift Coding Standards](/docs/coding_standards) (you can run 'make style', which ensures proper format for some languages).
* [*optional*] Verify that your change works on other platforms by adding a GitHub service hook to [Travis CI](http://docs.travis-ci.com/user/getting-started/#Step-one%3A-Sign-in) and [AppVeyor](http://www.appveyor.com/docs). You can use this technique to run the Thrift CI jobs in your account to check your changes before they are made public. Every GitHub pull request into Thrift will run the full CI build and test suite on your changes.
1. Squash your changes to a single commit. This maintains clean change history.
1. Commit and push changes to your branch (please use issue name and description as commit title, e.g. "THRIFT-9999: make it perfect"), with the affected languages on the next line of the description.
1. Use GitHub to create a pull request going from your branch to apache:master. Ensure that the Jira ticket number is at the beginning of the title of your pull request, same as the commit title.
1. Wait for other contributors or committers to review your new addition, and for a CI build to complete.
1. Wait for a committer to commit your patch. You can nudge the committers if necessary by sending a message to the [Apache Thrift mailing list](https://thrift.apache.org/mailing).
## If you want to build the project locally ##
For Windows systems, see our detailed instructions on the [CMake README](/build/cmake/README.md).
For Windows Native C++ builds, see our detailed instructions on the [WinCPP README](/build/wincpp/README.md).
For unix systems, see our detailed instructions on the [Docker README](/build/docker/README.md).
## If you want to review open issues... ##
1. Review the [GitHub Pull Request Backlog](https://github.com/apache/thrift/pulls). Code reviews are open to all.
2. Review the [Jira issue tracker](http://issues.apache.org/jira/browse/THRIFT). You can search for tickets relating to languages you are interested in or currently using with thrift, for example a Jira search (Issues -> Search For Issues) query of ``project = THRIFT AND component in ("Erlang - Library") and status not in (resolved, closed)`` will locate all the open Erlang Library issues.
## If you discovered a defect... ##
1. Check to see if the issue is already in the [Jira issue tracker](http://issues.apache.org/jira/browse/THRIFT).
1. If not, create a ticket describing the change you're proposing in the Jira issue tracker.
1. Contribute your code changes using the GitHub pull request method:
## Contributing via Patch ##
Some changes do not require a build, for example in documentation. For changes that are not code or build related, you can submit a patch on Jira for review. To create a patch from changes in your local directory:
git diff > ../THRIFT-NNNN.patch
then wait for contributors or committers to review your changes, and then for a committer to apply your patch.
## GitHub recipes for Pull Requests ##
Sometimes commmitters may ask you to take actions in your pull requests. Here are some recipes that will help you accomplish those requests. These examples assume you are working on Jira issue THRIFT-9999. You should also be familiar with the [upstream](https://help.github.com/articles/syncing-a-fork/) repository concept.
### Squash your changes ###
If you have not submitted a pull request yet, or if you have not yet rebased your existing pull request, you can squash all your commits down to a single commit. This makes life easier for the committers. If your pull request on GitHub has more than one commit, you should do this.
1. Use the command ``git log`` to identify how many commits you made since you began.
2. Use the command ``git rebase -i HEAD~N`` where N is the number of commits.
3. Leave "pull" in the first line.
4. Change all other lines from "pull" to "fixup".
5. All your changes are now in a single commit.
If you already have a pull request outstanding, you will need to do a "force push" to overwrite it since you changed your commit history:
git push -u origin THRIFT-9999 --force
A more detailed walkthrough of a squash can be found at [Git Ready](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html).
### Rebase your pull request ###
If your pull request has a conflict with master, it needs to be rebased:
git checkout THRIFT-9999
git rebase upstream master
(resolve any conflicts, make sure it builds)
git push -u origin THRIFT-9999 --force
### Fix a bad merge ###
If your pull request contains commits that are not yours, then you should use the following technique to fix the bad merge in your branch:
git checkout master
git pull upstream master
git checkout -b THRIFT-9999-take-2
git cherry-pick ...
(pick only your commits from your original pull request in ascending chronological order)
squash your changes to a single commit if there is more than one (see above)
git push -u origin THRIFT-9999-take-2:THRIFT-9999
This procedure will apply only your commits in order to the current master, then you will squash them to a single commit, and then you force push your local THRIFT-9999-take-2 into remote THRIFT-9999 which represents your pull request, replacing all the commits with the new one.

Some files were not shown because too many files have changed in this diff Show more