Commit graph

37 commits

Author SHA1 Message Date
shivrsrivastava
8454a6ebf3
Adding priority to the task (#140) 2022-10-12 21:46:07 -07:00
Renan I. Del Valle
c6a2a23ddb
Changing how constraints are handled internally (#115)
* Updating Changelog to reflect what's changing in 1.22.1

* Bug fix: Setting the same constraint multiple times is no longer allowed.

* Constraints map has been added to handle constraints being added to Aurora Jobs.

* Lowering timeout to avoid flaky test for bad payload timeout.

* Adding attributes to Mesos agents in order to test limits by constraint.



* Make two instances schedulable per zone in order to experience flaky behavior.
2020-01-15 08:21:12 -08:00
Renan DelValle
df8fc2fba1
Documentation and linting improvements (#108)
* Simplifying documentation for getting started: Removed outdated information about install Golang on different platforms and instead included a link to the official Golang website which has more up to date information. Instructions for installing docker-compose have also been added.

* Added documentation to all exported functions and structs.

* Unexported some structures and functions that were needlessly exported.

* Adding golang CI default configuration which can be useful while developing and may be turned on later in the CI.

* Moving build process in CI to xenial.

* Reducing line size. in some files and shadowing in some test cases.
2019-06-12 11:22:59 -07:00
Renan DelValle
6dc4bf93b9
Retry temporary errors by default (#107)
* Adding Aurora URL validator in order to handle scenarios where incomplete information is passed to the client. The client will do its best to guess the missing information such as protocol and port.

* Upgraded to testify 1.3.0.

* Added configuration to fail on a non-temporary error. This is reverting to the original behavior of the retry mechanism. However, this allows the user to opt to fail in a non-temporary error.
2019-06-11 11:47:14 -07:00
Renan DelValle
1a15c4a5aa
V1 CreateService and StartJobUpdate Timeout signal and cleanup (#105)
* Bumped up version to 1.21.1

* Moving admin functions to a new file. They are still part of the same pointer receiver type.

* Removing dead code and fixing some comments to add space between backslash and comment.

* Adding set up and tear down to run tests script. It sets up a pod, runs all tests, and then tears down the pod.

* Added `--rm` to run tests Mac script.

* Removing cookie jar from transport layer as it's not needed.

* Changing all error messages to start with a lower case letter. Changing some messages around to be more descriptive.

* Adding an argument to allow the retry mechanism to stop if a timeout has been encountered. This is useful for mutating API calls. Only StartUpdate and CreateService have enabled by default stop at timeout.

* Added 2 tests for when a call goes through despite the client timing out. One is with a good payload, one is with a bad payload.

* Updating changelog with information about the error type returned.

* Adding test for duplicate metadata.

* Refactored JobUpdateStatus monitor to use a new monitor called JobUpdateQuery. Update monitor will now still continue if it does not find an update to monitor. Furthermore, it has been optimized to reduce returning payloads from the scheduler as much as possible. This is through using the GetJobUpdateSummaries API instead of JobUpdateDetails and by including a the statuses we're searching for as part of the query.


* Added documentation as to how to handle a timeout on an API request.

* Optimized GetInstancesIds to create a copy of the JobKey being passed down in order to avoid unexpected behavior. Instead of setting every variable name separately, now a JobKey array is being created.
2019-05-05 11:46:22 -07:00
Renan DelValle
f7bd7cc20f
Bug fix for metadata duplicates as well as un-initialized GPU re… (#103)
* Fix for metadata duplicates as well.
* Fix for un-initialized GPU resource when creating a new job update.
2019-03-15 15:10:31 -07:00
Renan DelValle
773d842b03
Adding missing GPU to Job interface. 2019-03-05 11:43:50 -08:00
Renan DelValle
1f459dd56a
Adds support for Tier and SlaPolicy to the Job interface (#99)
* Adding parameter for Aurora so that we're able to run SLA aware updates with less than 20 instances. Lowered time it takes to run test by reducing watch time per instance as well.

* Reducing the number of instances and time for SLA aware instances in docker-compose set up.

* Adding another Mesos agent to the docker-compose setup.

* Huge thanks to @zircote for this contribution.
2019-02-20 16:36:50 -08:00
Renan DelValle
79fa7ba16d
Upgrading gorealis v1 to Thrift 0.12.0 code generation. End to end tests cleanup (#96)
* Ported all code from Thrift 0.9.3 to Thrift 0.12.0 while backporting some fixes from gorealis v2

* Removing git.apache.org dependency from Vendor folder as this dependency has migrated to github.

* Adding github.com thrift dependency back but now it points to github.com

* Removing unnecessary files from Thrift Vendor folder and adding them to .gitignore.

* Updating dep dependencies to include Thrift 0.12.0 from github.com

* Adding changelog.

* End to end tests: Adding coverage for killinstances.

*  End to end tests: Deleting instances after partition policy recovers them.

*  End to end tests: Adding more coverage to the realis API.

*  End to end tests: Allowing arguments to be passed to runTestMac so that '-run <test name>' can be passed in.

*  End to end tests: Reducing the resources used by CreateJob test.

*  End to end tests: Adding coverage for Pause and Resume update.

*   End to end tests: Removed checks for Aurora_OK response as that should always be handled by the error returned by the API. Changed names to be less verbose and repetitive.

*  End to end tests: Reducing watch time for instance running when creating service for reducing time it takes to run end to end test.
2019-02-20 11:11:46 -08:00
Renan DelValle
2f7015571c
Adding support for setting GPU as a resource. (#93)
* Adding support for setting GPU as a resource.
* Refactoring pulse update test.
2019-01-08 15:11:52 -08:00
Robert Allen
296af622d1 This adds the following function to the PartitionPolicy configuration to the Job interface (#91)
* Adding Partition Policy API
2018-12-20 14:38:06 -08:00
Renan DelValle
2eaa60f681
Support Drain SLA API (#88)
* Bringing thrift API up to date with Aurora 0.21.0.

* Adding support for SLA Drain Host API.
2018-11-16 11:41:09 -08:00
Renan DelValle
231793df71
Adding a separate function to add dedicated attributes. (#80)
Dedicated wrapper for "dedicated" constraints
2018-10-11 09:43:35 -07:00
Renan DelValle
8d445c1c77
Moving from govendor to dep, updated dependencies (#48)
* Moving from govendor to dep.

* Making the pull request template more friendly.

* Fixing akward space in PR template.

* goimports run on whole project using ` goimports -w $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./gen-go/*")`

source of command: https://gist.github.com/bgentry/fd1ffef7dbde01857f66
2018-01-07 13:13:47 -08:00
PRADYUMNA KAUSHIK
9631aa3aab Specify field names when initializing structs (#47)
* Added field names to struct initializations.
2017-12-23 10:33:42 -08:00
PRADYUMNA KAUSHIK
ff545e8aa6 Fixing semantic errors in docs/getting-started.md (#44)
* fixed semantics. Earlier mentioned that 'Pystachio does yet support...'. Changed now to mention 'Pystachio does not yet support...'

* fixed grammatical mistake.

* Ran gofmt on project.
2017-12-22 08:53:05 -08:00
Renan DelValle
e614e04f27
Code cleanup, added ability to attach logger, added CreateService api
* Code cleanup: Deleted multiple functions which have become stale. Removed cluster example as we replaced the need to create the Cluster object.

* Cleaned up ZK connection code by using the backoff function. Added a test to the end to end to test that we're getting the host correctly from ZK. Changed clusters test to be an outside package.

* Added LeaderFromZKURL test to end to end tests.

* Added logger to realisConfig so that users can attach their own Loggers to the client. Logger is an interface that shadows most popular logging libraries. Only Print, Println, and Printf are needed to be a realis.Logger type. Example in the client uses the std library log.

* Moved most fmt.Print* calls to be redirected to user provided logger. Logger by default is a no-op logger.

* Adding CreateService to realis interface. Uses the StartJobUpdate API to create services instead of the createJobs API.

* Bumping up version number inside client in anticipation of new release.
2017-11-30 12:02:50 -08:00
Renan DelValle
a1350c6d55 out with the old (address) in with the new (address) 2017-10-12 17:11:01 -07:00
Renan DelValle
f301affdd0 Initializing CPU, RAM, and DISK resources to have an address when a new job object is created so that were able to modify the values when constructing job updates. 2017-09-18 16:10:10 -07:00
Renan DelValle
13cc103faa Changing AuroraJob to *AuroraJob in order to not create new objects so that UpdateJob is able to change parameters. 2017-09-14 16:38:26 -07:00
Renan DelValle
d7efa913b1 Initial support for Mesos Containerizer 2017-02-23 14:21:04 -05:00
Renan DelValle
3bf2e8a831 Updating zookeeper dependency since logging problem has been solved in main repository. Go fmt run on project to tidy it up. 2016-11-02 20:41:43 -04:00
Renan DelValle
76b404e087 Submission of docker containers without an executor (aka running container as a task) is now supported 2016-11-02 15:24:06 -04:00
Renan DelValle
66a2868aab Adding support for configuring tasks with Docker Containers 2016-11-01 21:24:15 -04:00
Renan DelValle
15c2472ffd API for scheduling, descheduling, and starting cron jobs has been created. New response helper added for ScheduleStatusResult. 2016-09-30 01:24:49 -04:00
Renan DelValle
3a78e32e27 Changing location from where to get thrift bidnings to that it points to github location. Ran go fmt on the entire project. 2016-09-19 15:34:56 -04:00
Renan DelValle
928fc42fc2 Updated thrift API to the latest in the Aurora respository.
Added new monitors for watching number of instances get to a certain count using polling.
Added new commands to sample client which give some statistics.
2016-08-26 16:35:31 -07:00
Renan DelValle
dba42611d7 Added Job interface 2016-08-24 11:59:01 -07:00
Renan DelValle
e3c5a8c6be Changing prefix of un-named ports 2016-08-15 12:33:17 -07:00
Renan DelValle
c610974208 Moved Realis to be an interface for future testing classes. Removed AddURI API as AddURIs is able to do the same thing 2016-08-12 12:48:42 -07:00
Misho Krastev
80f11cf78b minor corrections to documentation and comments 2016-08-11 21:21:47 -07:00
Renan DelValle
403abcd684 Adding ability to add named ports to job configuration 2016-08-10 11:59:04 -07:00
Renan DelValle
bf890bf9b0 Updating documentation for release
Moving godoc reference button

changing casing of top title
2016-08-09 16:20:40 -07:00
Renan DelValle
a83956cb24 Minor style changes 2016-08-09 13:31:15 -07:00
Renan DelValle
93297f8c1a Shortening API 2016-08-02 13:06:36 -07:00
Renan DelValle
ab057e793b Shortening API by removing redundant words 2016-08-02 11:53:44 -07:00
Renan DelValle
327c6c369d Public release of gorealis 2016-08-02 11:42:00 -07:00