spdf/elektron
Archived
1
0
Fork 0
Code Issues 4 Pull requests 1 Projects Releases Packages Wiki Activity
563 commits 4 branches 0 tags 2.4 MiB
Commit graph

58 commits

Author SHA1 Message Date
Bhargavi Hanumant Alandikar
3543960689 Elektron Logging library (#16) 
switch to logrus for logging.

replaced old logging library with a wrapper around logrus.
We now just need to use the exported Log(...) and Logf(...) from the logging/
package that wraps around a set of loggers constituting a chain (following COR).
Loggers are configured using a YAML file that specifies the following.
1. enabled/disabled
2. whether the message should be logged on console.
3. filename extension.
4. minimum log level.

Retrofitted source code to now use the updated logging library.
Updated the documentation with information regarding the specification
of the log config file.

Currently, the log format in the config file is not adhered to. This is going to be
addressed in a future commit.
 2019-12-09 20:15:33 -05:00
PRADYUMNA KAUSHIK
9977251c14
workload validation before registering elektron. (#19) 
Added a utility to help with validating structs. This utility accepts
validators and runs them. If any of the validators fail, then the
error is wrapped with a given base message and returned.

Added validators for checking different attributes of a task
definition.

Added test code to test task validators.

Retrofitted scheduler.go to just log the task validation error and
terminate. If task validation does not report any error, then the
tasks are provided to the scheduler and elektron registers itself
with Mesos.
 2019-11-26 03:08:12 -05:00
PRADYUMNA KAUSHIK
e3caa90c31
Refactor imports, Add vendor/ as submodule (#5) 
refact imports to github.xxx. update go.mod + go.sum

* removed vendor/ folder

* Added vendor/ submodule.

Moved dependencies that need to be vendored into separate repository.
	- https://github.com/spdfg/elektron-vendor
Added vendor/ as a submodule using the below command.
	- git submodule add https://github.com/spdfg/elektron-vendor vendor
If wanting to use vendor, run the following commands after cloning
elektron.
1. git submodule init
2. git submodule update

* added instructions to clone vendor/ submodule.

* updated module to spdfg. Refactored imports
 2019-10-31 14:32:46 -04:00
Renan DelValle
291f4f3979
Adding GPLv3 Licensing. 2018-10-06 20:03:14 -07:00
Renan DelValle
9554bf2666
Runing goimports on project as well as fixing README so that headers render correctly. 2018-09-30 18:23:38 -07:00
Renan DelValle
45f9efa578
Changing imports from bitbucket to gitlab. 2018-09-30 17:56:14 -07:00
Pradyumna Kaushik
8b27f5547d Merged in fixSchedWindowSize (pull request #13) 
FixSchedWindowSize

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:18:50 -04:00
Pradyumna Kaushik
ae81125110 Merged in mapTaskDistrToSchedPolWhenSwitching (pull request #11) 
MapTaskDistrToSchedPolWhenSwitching

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:18:07 -04:00
Pradyumna Kaushik
85383da550 Merged in scheduleOnlySchedWindowTasks (pull request #6) 
ScheduleOnlySchedWindowTasks

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:07:22 -04:00
Pradyumna Kaushik
b58ffdbdd6 formatted code 2018-09-23 18:10:31 -04:00
Pradyumna Kaushik
435c4ca1bc fixed bug in schedWindow resizing strategy, where earlier it was stopping after checking if the first task in the queue can be scheduled in the next offer cycle. Changed baseScheduler to store the schedWindowResizingStrategy instead of the criteria. Retrofitted the scheduling policies to use the scheduling window resizing strategy directly from baseSchedRef. 2018-09-23 18:10:02 -04:00
Pradyumna Kaushik
3ebd7b0c7e added utility to compute the scheduling window. Right now there's only criteria on which this is determined -- fillNextOfferCycle. So, the schedWindow is the max number of tasks, that aren't yet scheduled, whose aggregate resource requirement is as close as possible to the resource available in the next round of resource offers. To be able to make the most use of the next offer cycle, one would need to perform a non-polynomial search of the TaskQueue and as this is computationally expensive, a linear search is performed on the TaskQueue. Retrofitted scheduling policies to also call utilities.schedUtils#schedWindowResizingStrategy#Apply before switching to a new scheduling policy. 2018-09-23 18:03:14 -04:00
Pradyumna Kaushik
9599588fb5 formatted code. 2018-09-23 17:49:08 -04:00
Pradyumna Kaushik
c31bf8db01 BUG Fix1: taskID creation in taskUtils#initTaskResourceRequirements. BUG Fix2: Converted value in utilities/trackResourceUsage#TrackResourceUsage#perHostResourceAvailability to be a pointer to ResourceCount. 2018-09-23 17:46:38 -04:00
Pradyumna Kaushik
657dc8df93 Track resource usage across the cluster. Created utility in utilities/ to track the total and the unused resources for each host in the cluster. Added utility to def/taskUtils.go to retrieve the resource requirement for a given taskID. Decoupled the code, to launch a list of tasks on a set of offerIDs, to schedulers/helpers.go and updated all the scheduling policies to call this function instead of directly calling mesos.SchedulerDriver#LaunchTasks. The resource availability of the cluster is updated at 2 stages -- 1. When the tasks are about to be launched (in schedulers/helpers.go#LaunchTasks), the scheduling policy switching logic will be able to adhere to the update in the resource availability due to the JUST launched tasks and 2. when a terminal status update is received for a task (in schedulers/base.go#statusUpdate). 2018-09-23 17:44:36 -04:00
Akash Kothawale
3b80af6d8f Fix: mesos-go imports 2018-09-23 17:31:03 -04:00
Pradyumna Kaushik
b807625b78 Fixed the comments to be capitalized at the start and also terminate with a period. 2017-09-28 15:36:47 -04:00
Pradyumna Kaushik
9897c983fe removed proactive-clusterwide power-capper. Retrofitted package names to use elektron and not electron 2017-09-26 00:18:35 -04:00
Abhishek Jain
806ac6fa86 Removed the import statement which was importing offerUtils from electron-archive. This results in build failure is electron-archive is a local package 2017-03-25 21:11:24 -04:00
Abhishek Jain
de7e73ff50 Changed a log message in the UpdateEnvironment(..) func. 2017-03-25 20:27:40 -04:00
Abhishek Jain
8eddad4e17 Changed the function name from AddHostIfNew(..) to UpdateEnvironment(..) based on a comment in the PR 2017-03-25 18:05:36 -04:00
Abhishek Jain
1915e589cd Made a tiny change in the logging message in AddHostIfNew(..) function 2017-03-24 17:03:29 -04:00
Abhishek Jain
abc718b554 Performed a git fmt 2017-03-24 16:34:17 -04:00
Abhishek Jain
10358b418b Added a space before { 2017-03-24 16:31:49 -04:00
Abhishek Jain
841c7d5ee8 Added AddHostIfNew(..) utility function which is responsible to populate the constants.Hosts and constants.PowerClasses 2017-03-24 16:27:14 -04:00
Abhishek Jain
26c49d4098 Restoring offerUtils. Had deleted this by mistake. 2017-03-23 22:09:51 -04:00
Abhishek Jain
9365c2e51d Changed the Hosts from list to a set of hosts using a map with key as hostname and value as an empty struct 2017-03-23 22:01:29 -04:00
Pradyumna Kaushik
64de61ac4e added function to initial list of Pairs from a map[string]float64. Removed unnecessary import. 2017-02-22 20:07:48 -05:00
Pradyumna Kaushik
de4e4c0141 removed unnecessary function. 2017-02-21 22:27:08 -05:00
Pradyumna Kaushik
13479e03a4 formatted files 2017-02-11 00:05:42 -05:00
Pradyumna Kaushik
1dcf416849 Resolved merge conflicts with master 2017-02-10 16:23:09 -05:00
Pradyumna Kaushik
814d16b54d added hostmismatch function to be called by all schedulers 2017-02-10 15:46:20 -05:00
Pradyumna Kaushik
6c62b5326f Added a HostMismatch(...) in offerUtils that checks whether a task's host requirement matches the host corresponding to the offer. Made sure all schedulers call takeOffer(...) that is defined in each scheduler, to maintain consistency. 2017-02-09 22:48:34 -05:00
Pradyumna Kaushik
fdcb401447 Made classMapWatts a commandLine option where one can enable and disable mapping of watts to powerclasses when accepting offers from Mesos. Removed the schedulers that were created solely for the classMapWatts feature. Retrofitted all schedulers to use the powerClass mapped watts attribute for a task, if classMapWatts has been enabled. Removed unnecessary functions and variables from constants.go. Removed unnecessary functions from utilities/utils.go. Fixed operator precendence issue with takeOffer(...) in some of the schedulers. Added TODO to decouple capping strategies from the schedulers completely. Added TODO to move all the common struct attributes in the schedulers into base.go. 2017-02-09 18:05:38 -05:00
Pradyumna Kaushik
e7166420dd add function to determine PowerClass of host in offer. This was earlier to be written in each scheduler when using classMapWatts. 2017-01-28 21:08:11 -05:00
Pradyumna Kaushik
5128c51b22 formatted code. 2017-01-28 19:45:47 -05:00
Pradyumna Kaushik
5a6edb802e fixed comment. 2017-01-28 19:43:12 -05:00
Pradyumna Kaushik
e63784b007 Created utilities/offerUtils that can can hold all the utility functions for Offers. 2017-01-28 19:32:54 -05:00
Pradyumna Kaushik
6d4446413d Created utilities/mesosUtils that can hold all the interactions with mesos. Moved longFilter and defaultFilter to mesosUtils. 2017-01-28 19:32:12 -05:00
Pradyumna Kaushik
2cd77a7ba8 Changed Window to ConsiderationWindow. 2017-01-28 18:29:00 -05:00
Renan DelValle
2678032c2c Go fmt on the entire project 2017-01-03 20:57:25 -05:00
Renan DelValle
147bbc8b5a Fixing commit history 2016-12-22 23:27:18 -05:00
Renan DelValle
71c889982f resolved merge conflicts 2016-12-22 23:17:01 -05:00
Pradyumna Kaushik
50d1d79051 fixed naming convensions to be camel cased. Reformatted the code. 2016-12-22 22:59:03 -05:00
Pradyumna Kaushik
6ae59d4be5 Revert "Removed a comment" 
This reverts commit fcdffb5c1006d7938d8be4aacad7ec5a7b78f20e.
 2016-12-22 22:59:03 -05:00
Pradyumna Kaushik
2a9c0eb81a Removed a comment 2016-12-22 22:59:03 -05:00
Pradyumna Kaushik
81d795f197 formatted the code 2016-12-22 22:59:03 -05:00
Pradyumna Kaushik
04d6d4b631 Utility data structures and functions. 2016-12-22 22:59:02 -05:00
Pradyumna Kaushik
9b5ac0bfa8 removed unnecessary variable 'numberOfElementsInWindow' and just used window.Len() instead 2016-12-22 22:59:01 -05:00
Pradyumna Kaushik
1b821735ed fixed bug in Remove() where the currentSum and numberOfTasksInWindow weren't getting updated. 2016-12-22 22:59:01 -05:00