Commit graph

109 commits

Author SHA1 Message Date
Sivaram Mothiki
dc327bebad change config for certs path (#57)
Bug fix. Changing hardcoded certificate to one provided by configuration object.
2018-03-02 15:21:45 -08:00
Renan DelValle
a43dc81ea8
Simplifying retry mechanism for Thrift Calls (#56)
* Deleting permament error as it doesn't make sense. Just return a plain old error and that will be considered permanent.

* Removing double closure at as it's unmaintainable and can be error prone. Separated back offs into a generic one and a thrift call specific one.

* ZK leader finder now returns a temporary error instead of constantly no leader found and quitting. It could be that the leader info is being propagated so it's worth trying another time.

* Adding more logging to the retry.

* Wrapping lock and unlock in an anonymous function so that we can use defer on unlock such that it is called in the case of a panic.
2018-02-15 15:16:39 -08:00
Renan DelValle
64948c3712
Backoff mechanism fix (#54)
* Fixing logic that can lead to nil error being returned and retry stopping early.

* Fixing possible code path that may lead to an incorrect nil error.
2018-02-06 12:44:27 -08:00
kkrishna
a6b077d1fd Aurora jobupdate functionality -- pause/resume/pulse api (#55)
* Adding GetJobs api

* Adding Aurora pause/resume/pulse api
2018-02-06 12:39:02 -08:00
kkrishna
8bd3957247 GetJobs api (#53)
* GetJobs API added
2018-01-27 10:33:55 -08:00
Renan DelValle
dbb08ded90
Simplifying KillJob API implementation (#52)
By submitting an empty set to the thrift call, the Aurora scheduler will look for active tasks itself.
2018-01-24 15:37:12 -08:00
Renan DelValle
a941bcb679
Thread safety, misc fixes, and refactoring (#51)
* Changing incorrect license in some source files.

* Changing CreateService to mimic CreateJob by setting the batch size to the instance count.

* Changing Getcerts to GetCerts to match the style of the rest of the codebase.

* Overhauled error handling. Backoff now recognizes temporary errors and continues to retry if it finds one.

* Changed thrift function call wrapper to be more explicitly named and to perform more safety checks.

* Moved Jitter function from realis to retry.

* API code is now more uniform and follows a certain template.

* Lock added whenever a thrift call is made or when a modification is done to the connection. Note that calling ReestablishConn externally may result in some race conditions. We will move to make this function private in the near future.

* Added test for Realis session thread safety. Tested ScheduleStatus monitor. Tested monitor timing out.

* Returning nil whenever there is an error return so that there are no ambiguities.

* Using defer with unlock so that the lock is still released if a panic is invoked.
2018-01-21 19:30:01 -08:00
Renan DelValle
b2ffb73183
Introducing temporary errors. Refactored reestablish connection code … (#50)
* Introducing temporary errors. 

* Refactored reestablish connection code to use NewClient.

* Added reestablish connection test to end to end tests.
2018-01-16 14:35:01 -08:00
PRADYUMNA KAUSHIK
9631aa3aab Specify field names when initializing structs (#47)
* Added field names to struct initializations.
2017-12-23 10:33:42 -08:00
Renan DelValle
c338c03355
Thrift API update and Pull Request template. (#43)
* Adding a Pull request template to serve as a reminder of what
to do before creating the pull request.

* Updating our thrift API to match changes made by Aurora.

* Update go bindings to match update thrift API.
2017-12-14 14:37:08 -08:00
Sivaram Mothiki
d4027bc95c make insecureskipverify configurable (#40)
* make inseucreskipverify configurable

* add insecure and certspath to configs

* add certs test

* add config support for client key and cert
2017-12-12 14:04:11 -08:00
Renan DelValle
e614e04f27
Code cleanup, added ability to attach logger, added CreateService api
* Code cleanup: Deleted multiple functions which have become stale. Removed cluster example as we replaced the need to create the Cluster object.

* Cleaned up ZK connection code by using the backoff function. Added a test to the end to end to test that we're getting the host correctly from ZK. Changed clusters test to be an outside package.

* Added LeaderFromZKURL test to end to end tests.

* Added logger to realisConfig so that users can attach their own Loggers to the client. Logger is an interface that shadows most popular logging libraries. Only Print, Println, and Printf are needed to be a realis.Logger type. Example in the client uses the std library log.

* Moved most fmt.Print* calls to be redirected to user provided logger. Logger by default is a no-op logger.

* Adding CreateService to realis interface. Uses the StartJobUpdate API to create services instead of the createJobs API.

* Bumping up version number inside client in anticipation of new release.
2017-11-30 12:02:50 -08:00
Sivaram Mothiki
72b746e431 use exponential back off func from realis lib (#39)
* use exponential back off func from realis lib

* remove exponential backoffs from monitors

* dont compare for retry errors
2017-11-04 15:06:26 -07:00
Renan DelValle
a1350c6d55 out with the old (address) in with the new (address) 2017-10-12 17:11:01 -07:00
Renan DelValle
bf354bcc0a Fixed bug with NewDefaultClientUsingUrl where backoff was not initialized leading to nil pointer error. 2017-09-28 17:14:55 -07:00
Renan DelValle
c03e8bf79c Added Maintenance status API 2017-09-28 16:32:17 -07:00
Renan DelValle
8fe3780949 Added end maintenance API which allows DRAINED hosts to be transitioned to ACTIVE. Fixed bug where payload error would never be returned if call failed due to a bad payload. 2017-09-27 14:02:43 -07:00
Renan DelValle
f59f0bbdc3 Setting retry for drain hosts to be a little more agressive for now. Added comments for go doc. DrainHosts now returns also returns a result type which will give API users the ability to operate directly on that object wihout using a response helper. 2017-09-25 15:45:40 -07:00
Renan DelValle
0d3126c468 New API to set hosts to DRAINING. Cleaned up some of the client code, and fixed a few error printing bugs. 2017-09-22 12:55:03 -07:00
Renan DelValle
719a121ed5 Bug fix: #30 NewRealisClient method returning <nil> error when zookeeper leader not found 2017-08-02 16:43:18 -07:00
Renan DelValle
99ea6aa094 Adding API call to get tasks without a fully hydrated object which results in less overhead 2017-07-24 15:45:24 -07:00
Mothiki
900f054633 check if the resposne schedule result is nil 2017-06-02 15:41:08 -07:00
Mukkavilli, Sunil
68d5b9c8fc Mustn't print the whole config object in logs, just printing the url 2017-04-18 16:35:27 -07:00
Mukkavilli, Sunil
89337f9c7f Merge branch 'master' of https://github.com/rdelval/gorealis 2017-04-18 16:10:58 -07:00
Kumar Krishna
e57dc98d65 add resiliency for LeaderFromZK and other fixes . 2017-04-06 23:15:44 -07:00
Kumar Krishna
48ca520eaa realisconfig refactor 2017-03-31 10:48:36 -07:00
Kumar Krishna
64dcf55be4 Merge branch 'master' of https://github.com/rdelval/gorealis 2017-03-30 18:17:50 -07:00
Kumar Krishna
b10df0603e gorealis config refactoring 2017-03-30 18:17:21 -07:00
Mukkavilli, Sunil
e6e4742f64 Merge branch 'master' of https://github.com/rdelval/gorealis 2017-03-24 15:55:48 -07:00
Renan DelValle
cb6100e690 Reverted change made to vendored thrift library which caused some issues when vendoring gorealis itself. Added a workaround to overcome the default httpPostClient in the thrift library defaulting to headers for the thrift JSON protocol 2017-03-23 19:18:23 -04:00
Kumar Krishna
9c4c2fce44 making some methods private. 2017-03-21 16:59:27 -07:00
Mukkavilli, Sunil
0d9b869583 Updating the client to not use instanceId 0 by default, instead get the instanceId from the list of active Instances 2017-03-21 12:52:29 -07:00
Kumar Krishna
3add32a585 gorealis resiliency 2017-03-20 22:34:45 -07:00
Kumar Krishna
d97e59b9e6 inital commit gorealis resiliency 2017-03-16 23:19:30 -07:00
Mothiki
f77379dd63 make gorealisCLient functions handled by client pointer 2017-03-10 15:50:26 -08:00
Mukkavilli, Sunil
40781fab82 Adding removeInstances method for a job -
-- This method takes the job key and the number of Instances to remove as parameters
2017-03-10 01:14:09 -08:00
smothiki
97fbec9eaf add function to get readonly scheduler client 2017-03-09 16:37:18 -08:00
Renan DelValle
10c12d5a13 Thrift Binary protocol is now a choice for users wishing to use it. Default Thrift protocol continues to be JSON. 2017-02-13 19:31:41 -05:00
Renan DelValle
5f155f4337 Moving from the Thrift JSON protocol to the Thrift Binary protocol by default. Realis config now holds transport and Protocol factory for impreoved flexibility. Renamed realis_test to reflect the true nature of the test along with a minor fix for an API change. 2017-02-10 19:23:20 -05:00
Kumar Krishna
f4fb52c237 New api GetTaskStatus 2016-11-15 22:24:07 -08:00
Kumar Krishna
8905233375 externalize timeout value 2016-11-14 13:27:25 -08:00
Renan DelValle
4c3bbc5079 Adding response code check to all api calls 2016-11-11 17:58:01 -05:00
Renan DelValle
3bf2e8a831 Updating zookeeper dependency since logging problem has been solved in main repository. Go fmt run on project to tidy it up. 2016-11-02 20:41:43 -04:00
Renan DelValle
15c2472ffd API for scheduling, descheduling, and starting cron jobs has been created. New response helper added for ScheduleStatusResult. 2016-09-30 01:24:49 -04:00
Renan DelValle
ca8b9359cf API for rolling back a job update has been integrated. 2016-09-30 00:44:38 -04:00
Renan DelValle
3fd957fe5c Updating library for compatibility with Aurora 0.16.0 final. Thrift API differs from main Aurora one due to TaskUpdateQuery fields resulting in errors when serialized by thrift as non-optionals. 2016-09-29 20:45:24 -04:00
Renan DelValle
3a78e32e27 Changing location from where to get thrift bidnings to that it points to github location. Ran go fmt on the entire project. 2016-09-19 15:34:56 -04:00
Renan DelValle
928fc42fc2 Updated thrift API to the latest in the Aurora respository.
Added new monitors for watching number of instances get to a certain count using polling.
Added new commands to sample client which give some statistics.
2016-08-26 16:35:31 -07:00
Renan DelValle
01b700554a Added the ability to monitor job updates.
Added the ability to kill and restart specific instances.
Fixed incorrect documentation on using-the-sample-client.
Added helper functions under the response package to extract fields from
aurora.Response.
2016-08-25 18:56:55 -07:00
Renan DelValle
bb3b26340d Ran go fmt on project. Changed FechTaskConfig to only fetch configurations in an existing state 2016-08-24 17:21:59 -07:00