spdf/elektron
Archived
1
0
Fork 0
Code Issues 4 Pull requests 1 Projects Releases Packages Wiki Activity
563 commits 4 branches 0 tags 2.4 MiB
Commit graph

43 commits

Author SHA1 Message Date
Bhargavi Hanumant Alandikar
3543960689 Elektron Logging library (#16) 
switch to logrus for logging.

replaced old logging library with a wrapper around logrus.
We now just need to use the exported Log(...) and Logf(...) from the logging/
package that wraps around a set of loggers constituting a chain (following COR).
Loggers are configured using a YAML file that specifies the following.
1. enabled/disabled
2. whether the message should be logged on console.
3. filename extension.
4. minimum log level.

Retrofitted source code to now use the updated logging library.
Updated the documentation with information regarding the specification
of the log config file.

Currently, the log format in the config file is not adhered to. This is going to be
addressed in a future commit.
 2019-12-09 20:15:33 -05:00
PRADYUMNA KAUSHIK
9977251c14
workload validation before registering elektron. (#19) 
Added a utility to help with validating structs. This utility accepts
validators and runs them. If any of the validators fail, then the
error is wrapped with a given base message and returned.

Added validators for checking different attributes of a task
definition.

Added test code to test task validators.

Retrofitted scheduler.go to just log the task validation error and
terminate. If task validation does not report any error, then the
tasks are provided to the scheduler and elektron registers itself
with Mesos.
 2019-11-26 03:08:12 -05:00
PRADYUMNA KAUSHIK
e3caa90c31
Refactor imports, Add vendor/ as submodule (#5) 
refact imports to github.xxx. update go.mod + go.sum

* removed vendor/ folder

* Added vendor/ submodule.

Moved dependencies that need to be vendored into separate repository.
	- https://github.com/spdfg/elektron-vendor
Added vendor/ as a submodule using the below command.
	- git submodule add https://github.com/spdfg/elektron-vendor vendor
If wanting to use vendor, run the following commands after cloning
elektron.
1. git submodule init
2. git submodule update

* added instructions to clone vendor/ submodule.

* updated module to spdfg. Refactored imports
 2019-10-31 14:32:46 -04:00
Pradyumna Kaushik
bac60e872a Unit testing for def/ module. 
Added unit tests to test code in def/ module.
 2019-10-12 06:48:45 +00:00
Renan DelValle
291f4f3979
Adding GPLv3 Licensing. 2018-10-06 20:03:14 -07:00
Renan DelValle
9554bf2666
Runing goimports on project as well as fixing README so that headers render correctly. 2018-09-30 18:23:38 -07:00
Renan DelValle
45f9efa578
Changing imports from bitbucket to gitlab. 2018-09-30 17:56:14 -07:00
Pradyumna Kaushik
66c19b53c9 Merged in differentSwitchingMechanisms (pull request #14) 
DifferentSwitchingMechanisms

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:21:52 -04:00
Pradyumna Kaushik
ae81125110 Merged in mapTaskDistrToSchedPolWhenSwitching (pull request #11) 
MapTaskDistrToSchedPolWhenSwitching

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:18:07 -04:00
Pradyumna Kaushik
85383da550 Merged in scheduleOnlySchedWindowTasks (pull request #6) 
ScheduleOnlySchedWindowTasks

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:07:22 -04:00
Pradyumna Kaushik
6c77aa777e Merged in resourceUsageTracking (pull request #2) 
ResourceUsageTracking

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
Approved-by: Pradyumna Kaushik <pkaushi1@binghamton.edu>
 2018-09-23 18:36:07 -04:00
Pradyumna Kaushik
9599588fb5 formatted code. 2018-09-23 17:49:08 -04:00
Pradyumna Kaushik
c31bf8db01 BUG Fix1: taskID creation in taskUtils#initTaskResourceRequirements. BUG Fix2: Converted value in utilities/trackResourceUsage#TrackResourceUsage#perHostResourceAvailability to be a pointer to ResourceCount. 2018-09-23 17:46:38 -04:00
Pradyumna Kaushik
657dc8df93 Track resource usage across the cluster. Created utility in utilities/ to track the total and the unused resources for each host in the cluster. Added utility to def/taskUtils.go to retrieve the resource requirement for a given taskID. Decoupled the code, to launch a list of tasks on a set of offerIDs, to schedulers/helpers.go and updated all the scheduling policies to call this function instead of directly calling mesos.SchedulerDriver#LaunchTasks. The resource availability of the cluster is updated at 2 stages -- 1. When the tasks are about to be launched (in schedulers/helpers.go#LaunchTasks), the scheduling policy switching logic will be able to adhere to the update in the resource availability due to the JUST launched tasks and 2. when a terminal status update is received for a task (in schedulers/base.go#statusUpdate). 2018-09-23 17:44:36 -04:00
Akash Kothawale
3b80af6d8f Fix: mesos-go imports 2018-09-23 17:31:03 -04:00
Pradyumna Kaushik
b807625b78 Fixed the comments to be capitalized at the start and also terminate with a period. 2017-09-28 15:36:47 -04:00
Pradyumna Kaushik
59266d207d retrofitted to use the generic task sorter. 2017-09-26 00:37:12 -04:00
Pradyumna Kaushik
9897c983fe removed proactive-clusterwide power-capper. Retrofitted package names to use elektron and not electron 2017-09-26 00:18:35 -04:00
Pradyumna Kaushik
3a9db01fa0 formatted files 2017-08-26 22:33:06 -04:00
Pradyumna Kaushik
8bc408dea1 fixed comments. Removed an old comment that no longer made sense. 2017-08-26 15:43:09 -04:00
Pradyumna Kaushik
9e620eaede changed def.TaskSorter(...) to def.SortTasks(...) for semantics. Now, def.SortTasks(...) also sorts the tasks instead of returning just a func (i, j int) bool {}. Removed TODO from the README associated with this PR 2017-08-26 15:39:45 -04:00
Pradyumna Kaushik
0869bea2d8 Used closures instead of strings for the generic sorter. This removed the need for reflection. 2017-08-26 15:19:30 -04:00
Pradyumna Kaushik
3c65bdf02e changed the scope of TaskResourceNames and SortCriteria to be package private. 2017-08-23 20:10:33 -04:00
Pradyumna Kaushik
f29b7f51a9 Added a generic task sorter utility. This allows for an array of tasks to be sorted based on any resource. The possible resources by which an array of tasks can be sorted currently are listed in def/sortingCriteria.go. One can add more to this if required. 2017-08-23 19:35:19 -04:00
Pradyumna Kaushik
e1350838f3 Merge branch 'kmeansTaskClassification' of https://bitbucket.org/sunybingcloud/electron into kmeansTaskClassification 2017-08-22 13:03:17 -04:00
Pradyumna Kaushik
f4459c8cbf Consolidated the ClassifyTasks(...) functions from topHeavy and bottomHeavy and added the function to def/taskUtils.go. Added TODOs for refining the means by which the kmeans classified clusters were sorted. 2017-08-22 13:00:40 -04:00
Pradyumna Kaushik
e3569e95ce fixed comments to be a little more meaningful. 2017-04-30 16:48:38 -04:00
Pradyumna Kaushik
1b15bb14e7 Resolved merge conflicts with the master. Also, added TODO in README to use Go1.8 techniques. 2017-04-21 17:17:22 -04:00
Abhishek Jain
f85ed944f4 Changed the loop as now we are looping over a map and not a slice. 2017-03-23 22:13:29 -04:00
Pradyumna Kaushik
74eb616a72 the observation that is to be considered for the classification of a task can now be given as a function that takes the task and returns the observations as a slice of float64. Retrofitted the other functions in taskUtils to make use of this feature. 2017-02-25 19:57:01 -05:00
Pradyumna Kaushik
e54697b0dc Added a task utility to be able to cluster tasks into N clusters based on their watts resource requirements. Electron now compatible with Go1.8 and no longer with 1.7. Added TODOs. 2017-02-25 15:43:32 -05:00
Pradyumna Kaushik
9dddc38cad created a utility for tasks that allows for KMeans clustering based on watts resource requirement 2017-02-24 20:52:59 -05:00
Pradyumna Kaushik
fdcb401447 Made classMapWatts a commandLine option where one can enable and disable mapping of watts to powerclasses when accepting offers from Mesos. Removed the schedulers that were created solely for the classMapWatts feature. Retrofitted all schedulers to use the powerClass mapped watts attribute for a task, if classMapWatts has been enabled. Removed unnecessary functions and variables from constants.go. Removed unnecessary functions from utilities/utils.go. Fixed operator precendence issue with takeOffer(...) in some of the schedulers. Added TODO to decouple capping strategies from the schedulers completely. Added TODO to move all the common struct attributes in the schedulers into base.go. 2017-02-09 18:05:38 -05:00
Pradyumna Kaushik
b838c53c6d added CPUSorter to task to be able to sort tasks based on CPU requirement. 2017-02-05 14:49:07 -05:00
Renan DelValle
2678032c2c Go fmt on the entire project 2017-01-03 20:57:25 -05:00
Renan DelValle
312728a2f6 Adding node class mapping for Watts as a resource in binpack 2016-12-23 21:04:15 -05:00
Pradyumna Kaushik
6289b8954d Prevented the use of reflect and instead compared the TaskIDs of the tasks. 2016-12-22 22:59:03 -05:00
Pradyumna Kaushik
50d1d79051 fixed naming convensions to be camel cased. Reformatted the code. 2016-12-22 22:59:03 -05:00
Pradyumna Kaushik
cd644bbf69 Formatted the code 2016-12-22 22:59:02 -05:00
Pradyumna Kaushik
0f8a2d3f0c fixed a an error. 2016-12-22 22:59:02 -05:00
Pradyumna Kaushik
0c53bb386f Added TaskID as a field. Added a function UpdateHost() to update the host on which the task runs. Added a setter for TaskID. Added a comparator called Compare() that compares to instances of Task 2016-12-22 22:59:02 -05:00
Renan DelValle
617fb8ac0f Sorting based on watts value added. bin packing based on watts is almost complete. 2016-12-22 22:59:02 -05:00
Renan DelValle
c2e2b7e554 Moved schedulers from the main programs to schedulers package. Can now choose different scheduelrs to use. Work on code sharing between schedulers remains to be done. 2016-12-22 22:59:02 -05:00