spdf/elektron
Archived
1
0
Fork 0
Code Issues 4 Pull requests 1 Projects Releases Packages Wiki Activity
539 commits 4 branches 0 tags 2.4 MiB
Commit graph

33 commits

Author SHA1 Message Date
Renan DelValle
291f4f3979
Adding GPLv3 Licensing. 2018-10-06 20:03:14 -07:00
Pradyumna Kaushik
4637355721 Reorganized code. 
Building the scheduler in phases. To be able to do this, increased
visibility of schedulers/helpers.go#schedulerOptions.
Refactored dependent code.
 2018-10-04 19:27:13 -04:00
Pradyumna Kaushik
8e87bcb439 Fixed import alias prefix. 
Changed the prefix to import aliases to be 'elek' instead of 'elec'.
 2018-10-04 13:45:31 -04:00
Renan DelValle
9554bf2666
Runing goimports on project as well as fixing README so that headers render correctly. 2018-09-30 18:23:38 -07:00
Renan DelValle
45f9efa578
Changing imports from bitbucket to gitlab. 2018-09-30 17:56:14 -07:00
Pradyumna Kaushik
66c19b53c9 Merged in differentSwitchingMechanisms (pull request #14) 
DifferentSwitchingMechanisms

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:21:52 -04:00
Pradyumna Kaushik
8b27f5547d Merged in fixSchedWindowSize (pull request #13) 
FixSchedWindowSize

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:18:50 -04:00
Pradyumna Kaushik
f1c6adb05b Merged in measureClassificationOverhead (pull request #12) 
MeasureClassificationOverhead

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:18:30 -04:00
Pradyumna Kaushik
ae81125110 Merged in mapTaskDistrToSchedPolWhenSwitching (pull request #11) 
MapTaskDistrToSchedPolWhenSwitching

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:18:07 -04:00
Pradyumna Kaushik
0f305ab796 Merged in updateRunningOnTaskRunningStatus (pull request #9) 
critical: baseScheduler#Running wasn't getting update properly.

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:12:50 -04:00
Pradyumna Kaushik
b877d31cb8 Merged in schedPolSwitchConfigFile (pull request #8) 
SchedPolSwitchConfigFile

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:10:23 -04:00
Pradyumna Kaushik
85383da550 Merged in scheduleOnlySchedWindowTasks (pull request #6) 
ScheduleOnlySchedWindowTasks

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
 2018-09-23 19:07:22 -04:00
Akash Kothawale
b1bd21f730 Merged in akash/LogPolicySwitch (pull request #4) 
Scheduling Policy Switch Logger

Approved-by: Pradyumna Kaushik <pkaushi1@binghamton.edu>
 2018-09-23 18:57:24 -04:00
Pradyumna Kaushik
6c77aa777e Merged in resourceUsageTracking (pull request #2) 
ResourceUsageTracking

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
Approved-by: Pradyumna Kaushik <pkaushi1@binghamton.edu>
 2018-09-23 18:36:07 -04:00
Pradyumna Kaushik
f041e6668b added a baseSchedPolicyState struct in schedulers/schedPolicy.go. This struct would store information common to scheduling policies. Added member called numTasksScheduled to baseSchedPolicyState that would keep count of the number of tasks that the current scheduling policy has scheduled. Moved the logic to switch (currently performing a random switch) to baseSchedPolicyState#switchIfNecessary(...) and retrofitted all scheduling policies to call this instead of inlining the code in each of them. 2018-09-23 18:28:11 -04:00
Pradyumna Kaushik
5a28f8539a formatted code. 2018-09-23 18:23:37 -04:00
Akash Kothawale
3d5bb772fb Fix: Mutex lock handling issues with task running 2018-09-23 18:15:52 -04:00
Akash Kothawale
46be28ef8d pcp: CPU/MEM utilization & task share variance per node 2018-09-23 18:13:28 -04:00
Pradyumna Kaushik
b58ffdbdd6 formatted code 2018-09-23 18:10:31 -04:00
Pradyumna Kaushik
435c4ca1bc fixed bug in schedWindow resizing strategy, where earlier it was stopping after checking if the first task in the queue can be scheduled in the next offer cycle. Changed baseScheduler to store the schedWindowResizingStrategy instead of the criteria. Retrofitted the scheduling policies to use the scheduling window resizing strategy directly from baseSchedRef. 2018-09-23 18:10:02 -04:00
Pradyumna Kaushik
3ebd7b0c7e added utility to compute the scheduling window. Right now there's only criteria on which this is determined -- fillNextOfferCycle. So, the schedWindow is the max number of tasks, that aren't yet scheduled, whose aggregate resource requirement is as close as possible to the resource available in the next round of resource offers. To be able to make the most use of the next offer cycle, one would need to perform a non-polynomial search of the TaskQueue and as this is computationally expensive, a linear search is performed on the TaskQueue. Retrofitted scheduling policies to also call utilities.schedUtils#schedWindowResizingStrategy#Apply before switching to a new scheduling policy. 2018-09-23 18:03:14 -04:00
Akash Kothawale
6f0f3788b9 sched: Make baseScheduler publicly accessible 2018-09-23 17:55:12 -04:00
Pradyumna Kaushik
9599588fb5 formatted code. 2018-09-23 17:49:08 -04:00
Pradyumna Kaushik
657dc8df93 Track resource usage across the cluster. Created utility in utilities/ to track the total and the unused resources for each host in the cluster. Added utility to def/taskUtils.go to retrieve the resource requirement for a given taskID. Decoupled the code, to launch a list of tasks on a set of offerIDs, to schedulers/helpers.go and updated all the scheduling policies to call this function instead of directly calling mesos.SchedulerDriver#LaunchTasks. The resource availability of the cluster is updated at 2 stages -- 1. When the tasks are about to be launched (in schedulers/helpers.go#LaunchTasks), the scheduling policy switching logic will be able to adhere to the update in the resource availability due to the JUST launched tasks and 2. when a terminal status update is received for a task (in schedulers/base.go#statusUpdate). 2018-09-23 17:44:36 -04:00
Akash Kothawale
3b80af6d8f Fix: mesos-go imports 2018-09-23 17:31:03 -04:00
Pradyumna Kaushik
065705d480 Merged in experimentation/schedPolicySwitcher (pull request #1) 
Experimentation/schedPolicySwitcher
1. Initial commit for consolidated loggers using observer pattern.
2. class factory for schedulers.
3. Using the scheduling policy class factory in schedulers/store.go and the scheduler builder helpers in schedulers/helpers.go, feature to be able to be able to plug a scheduling policy of your choice from the command line (right now only first-fit and bin-packing are possible. Will be updating the class factory to include other scheduling policies as well.
4. Removed TODO for using generic task sorters. Modified TODO for a config file input to run electron.
5. Added other schedulers to the factory
6. Partially retrofitted the other scheduling policies to use the logging library.
7. Retrofitted extrema and progressive to use the consolidated logging library. Fixed parameter issue with s.base.Disconnected(). Formatted project
8. Move statusUpdate(...) into base.go to remove redundant code.
9. Converted the baseScheduler into a state machine where the state is a scheduling policy that defines an approach to consume resource offers.
10. Added another command line argument to be used to enable switching of scheduling policies. Retrofitted scheduling policies to switch only if the particular feature has been enabled.

changed argument to coLocated(...) to take base type rather than ElectronScheduler type. Also, prepended the prefix to the directory of the logs so that it would be easier to determine what the files in a directory correspond to without viewing the contents of the directory.
Defined methods in ElectronScheduler. Each of these methods corresponds to a type of log that an ElectronScheduler would make. Each of these methods would need to be implemented by the scheduling policy.

Electron has only one scheduler that implements the mesos scheduler interface. All the scheduling policies are just different implementations of ways to consume mesos resource offers. Retrofitted scheduling policies to now embed SchedPolicyState instead of baseScheduler.

Approved-by: Pradyumna Kaushik <pkaushi1@binghamton.edu>
 2018-09-23 17:19:28 -04:00
Pradyumna Kaushik
b807625b78 Fixed the comments to be capitalized at the start and also terminate with a period. 2017-09-28 15:36:47 -04:00
Pradyumna Kaushik
04f24beac5 scheduling policies pluggable from commandline 2017-09-26 13:17:47 -04:00
Pradyumna Kaushik
9897c983fe removed proactive-clusterwide power-capper. Retrofitted package names to use elektron and not electron 2017-09-26 00:18:35 -04:00
Pradyumna Kaushik
d4b1a85f79 resolved merge conflict with master. Formatted file. 2017-02-11 14:27:33 -05:00
Pradyumna Kaushik
9dc5bdada2 Moved all the common scheduler attributes into base.go 2017-02-09 20:27:18 -05:00
Renan DelValle
841c5e0392 Minor comments and style changes 2017-01-06 18:53:03 -05:00
Renan DelValle
b636ff490c Using anonymous and type promotion to code share more efficiently 2017-01-03 20:56:55 -05:00