spdf/elektron
Archived
1
0
Fork 0
Code Issues 4 Pull requests 1 Projects Releases Packages Wiki Activity
464 commits 4 branches 0 tags 2.4 MiB
Commit graph

205 commits

Author SHA1 Message Date
Pradyumna Kaushik
5a28f8539a formatted code. 2018-09-23 18:23:37 -04:00
Akash Kothawale
3d5bb772fb Fix: Mutex lock handling issues with task running 2018-09-23 18:15:52 -04:00
Akash Kothawale
46be28ef8d pcp: CPU/MEM utilization & task share variance per node 2018-09-23 18:13:28 -04:00
Pradyumna Kaushik
b58ffdbdd6 formatted code 2018-09-23 18:10:31 -04:00
Pradyumna Kaushik
435c4ca1bc fixed bug in schedWindow resizing strategy, where earlier it was stopping after checking if the first task in the queue can be scheduled in the next offer cycle. Changed baseScheduler to store the schedWindowResizingStrategy instead of the criteria. Retrofitted the scheduling policies to use the scheduling window resizing strategy directly from baseSchedRef. 2018-09-23 18:10:02 -04:00
Pradyumna Kaushik
3ebd7b0c7e added utility to compute the scheduling window. Right now there's only criteria on which this is determined -- fillNextOfferCycle. So, the schedWindow is the max number of tasks, that aren't yet scheduled, whose aggregate resource requirement is as close as possible to the resource available in the next round of resource offers. To be able to make the most use of the next offer cycle, one would need to perform a non-polynomial search of the TaskQueue and as this is computationally expensive, a linear search is performed on the TaskQueue. Retrofitted scheduling policies to also call utilities.schedUtils#schedWindowResizingStrategy#Apply before switching to a new scheduling policy. 2018-09-23 18:03:14 -04:00
Akash Kothawale
6f0f3788b9 sched: Make baseScheduler publicly accessible 2018-09-23 17:55:12 -04:00
Pradyumna Kaushik
9599588fb5 formatted code. 2018-09-23 17:49:08 -04:00
Pradyumna Kaushik
c31bf8db01 BUG Fix1: taskID creation in taskUtils#initTaskResourceRequirements. BUG Fix2: Converted value in utilities/trackResourceUsage#TrackResourceUsage#perHostResourceAvailability to be a pointer to ResourceCount. 2018-09-23 17:46:38 -04:00
Pradyumna Kaushik
657dc8df93 Track resource usage across the cluster. Created utility in utilities/ to track the total and the unused resources for each host in the cluster. Added utility to def/taskUtils.go to retrieve the resource requirement for a given taskID. Decoupled the code, to launch a list of tasks on a set of offerIDs, to schedulers/helpers.go and updated all the scheduling policies to call this function instead of directly calling mesos.SchedulerDriver#LaunchTasks. The resource availability of the cluster is updated at 2 stages -- 1. When the tasks are about to be launched (in schedulers/helpers.go#LaunchTasks), the scheduling policy switching logic will be able to adhere to the update in the resource availability due to the JUST launched tasks and 2. when a terminal status update is received for a task (in schedulers/base.go#statusUpdate). 2018-09-23 17:44:36 -04:00
Pradyumna Kaushik
6cd61ed18b comment cleanup. 2018-09-23 17:33:34 -04:00
Akash Kothawale
3b80af6d8f Fix: mesos-go imports 2018-09-23 17:31:03 -04:00
Pradyumna Kaushik
065705d480 Merged in experimentation/schedPolicySwitcher (pull request #1) 
Experimentation/schedPolicySwitcher
1. Initial commit for consolidated loggers using observer pattern.
2. class factory for schedulers.
3. Using the scheduling policy class factory in schedulers/store.go and the scheduler builder helpers in schedulers/helpers.go, feature to be able to be able to plug a scheduling policy of your choice from the command line (right now only first-fit and bin-packing are possible. Will be updating the class factory to include other scheduling policies as well.
4. Removed TODO for using generic task sorters. Modified TODO for a config file input to run electron.
5. Added other schedulers to the factory
6. Partially retrofitted the other scheduling policies to use the logging library.
7. Retrofitted extrema and progressive to use the consolidated logging library. Fixed parameter issue with s.base.Disconnected(). Formatted project
8. Move statusUpdate(...) into base.go to remove redundant code.
9. Converted the baseScheduler into a state machine where the state is a scheduling policy that defines an approach to consume resource offers.
10. Added another command line argument to be used to enable switching of scheduling policies. Retrofitted scheduling policies to switch only if the particular feature has been enabled.

changed argument to coLocated(...) to take base type rather than ElectronScheduler type. Also, prepended the prefix to the directory of the logs so that it would be easier to determine what the files in a directory correspond to without viewing the contents of the directory.
Defined methods in ElectronScheduler. Each of these methods corresponds to a type of log that an ElectronScheduler would make. Each of these methods would need to be implemented by the scheduling policy.

Electron has only one scheduler that implements the mesos scheduler interface. All the scheduling policies are just different implementations of ways to consume mesos resource offers. Retrofitted scheduling policies to now embed SchedPolicyState instead of baseScheduler.

Approved-by: Pradyumna Kaushik <pkaushi1@binghamton.edu>
 2018-09-23 17:19:28 -04:00
Pradyumna Kaushik
cb71153362 Sorting the tasks in non-decreasing order of watts for max-greedymins and max-min. Had unintentionally removed the sorting of tasks for max-greedymins and max-mins. 2018-01-18 18:30:27 -05:00
Pradyumna Kaushik
ad9f67a2a0 moved documentation of scheduling policies and power-capping strategies into a documentation folder called docs/ 2017-09-28 15:53:36 -04:00
Pradyumna Kaushik
4051aa1fb3 changed the named of the scheduling policy documentation to SchedulingPolicies.md. Also, added a documentation for power-capping strategies called PowerCappingStrategies.md 2017-09-28 15:51:22 -04:00
Pradyumna Kaushik
b807625b78 Fixed the comments to be capitalized at the start and also terminate with a period. 2017-09-28 15:36:47 -04:00
Pradyumna Kaushik
971f6833c3 modified schedulers/README.md to now use Elektron instead of Electron. Also, changed the content to just display the contained scheduling policies with an explaination for each of them. 2017-09-26 13:43:23 -04:00
Pradyumna Kaushik
04f24beac5 scheduling policies pluggable from commandline 2017-09-26 13:17:47 -04:00
Pradyumna Kaushik
60e5e0ecc1 added max-min scheduling policy 2017-09-26 01:14:39 -04:00
Pradyumna Kaushik
3fa4f3d0e6 changed commandline argument descriptions. Changed the names of the scheduling policies and the structs. 2017-09-26 01:05:18 -04:00
Pradyumna Kaushik
59266d207d retrofitted to use the generic task sorter. 2017-09-26 00:37:12 -04:00
Pradyumna Kaushik
9897c983fe removed proactive-clusterwide power-capper. Retrofitted package names to use elektron and not electron 2017-09-26 00:18:35 -04:00
Pradyumna Kaushik
1c4b6f0f01 refined README and removed unwanted scheduling policies. 2017-09-26 00:05:19 -04:00
Pradyumna Kaushik
235ed189d5 Moved the TODO, for the refinement of the cluster sorting algorithm, to the main README file. 2017-08-22 13:09:05 -04:00
Pradyumna Kaushik
f4459c8cbf Consolidated the ClassifyTasks(...) functions from topHeavy and bottomHeavy and added the function to def/taskUtils.go. Added TODOs for refining the means by which the kmeans classified clusters were sorted. 2017-08-22 13:00:40 -04:00
Pradyumna Kaushik
1b15bb14e7 Resolved merge conflicts with the master. Also, added TODO in README to use Go1.8 techniques. 2017-04-21 17:17:22 -04:00
Abhishek Jain
e01c7b1b1d Retrofitted all the schedulers to call UpdateEnvironment(..) function instead of the old named function i.e AddHostIfNew(..) 2017-03-25 18:06:39 -04:00
Abhishek Jain
6665ff4776 Fixed a bug. 2017-03-24 16:36:42 -04:00
Abhishek Jain
abc718b554 Performed a git fmt 2017-03-24 16:34:17 -04:00
Abhishek Jain
b4f9a989cb Retrofitted all the schedulers to now call the offerutils.AddHostIfNew(..) utility function which will dynamically populate the constants.Hosts and constants.PowerClasses. 2017-03-24 16:28:49 -04:00
Abhishek Jain
aed4fd1073 Retrofitted all schedulers to now pick the hostname from the offer and add it to constants.Hosts 2017-03-23 22:16:05 -04:00
Pradyumna Kaushik
dd3019735d Merge branch 'master' of https://bitbucket.org/sunybingcloud/electron into progressiveExtrema 2017-03-18 19:54:37 -04:00
Pradyumna Kaushik
190b395bc3 fixed bug to return []float64{...} as observations rather than one value. 2017-03-13 16:44:52 -04:00
Pradyumna Kaushik
34ab753491 Used the KMeans classification to classify the tasks. 2017-03-13 16:38:53 -04:00
Pradyumna Kaushik
668ae586fe Merge branch 'newPowerClasses' of https://bitbucket.org/sunybingcloud/electron into kmeansTaskClassification 2017-03-10 17:26:39 -05:00
Pradyumna Kaushik
505ccc0471 reclassified nodes in the cluster into 4 power classes instead of the original 3. Retrofitted the hybrid schedulers TopHeavy and BottomHeavy to comply with the change classification of nodes" 2017-03-10 17:03:54 -05:00
Pradyumna Kaushik
1897678452 Merge branch 'master' of https://bitbucket.org/sunybingcloud/electron into kmeansTaskClassification 2017-02-28 15:28:49 -05:00
Pradyumna Kaushik
9eab5db2ea Merge branch 'master' of https://bitbucket.org/sunybingcloud/electron into progressiveExtrema 2017-02-28 15:28:06 -05:00
Pradyumna Kaushik
c18dd0fa70 formatted 2017-02-27 19:03:36 -05:00
Pradyumna Kaushik
ef8a315e90 fixed aggregate check for BinPacking. This had been broken due to a previous merge. 2017-02-27 18:37:53 -05:00
Pradyumna Kaushik
ef6f74cd81 removed TODO for consolidating common scheduler struct members into base.go. 2017-02-20 22:42:07 -05:00
Pradyumna Kaushik
d42b7a3a3b changed the type of percentage in rapl.Cap(...) from int to float64. Retrofitted power-capping strategies to cap using a float64 value instead of an int. Moved common functions in loganddynamiccap.go and logAndProgressiveExtrema.go into pcp/utils.go. New power-capping strategy that builds on top of extrema, where it caps the victims at different until it can't cap further, in which case it starts uncapping them in the reverse order of capping. 2017-02-15 19:22:56 -05:00
Pradyumna Kaushik
d4b1a85f79 resolved merge conflict with master. Formatted file. 2017-02-11 14:27:33 -05:00
Pradyumna Kaushik
02fede7184 resolved merge conflict with master. Also, changed the name of the constructor for BPSWMaxMin from NewBPMaxMinWatts to NewBPSWMaxMinWatts 2017-02-11 14:26:27 -05:00
Pradyumna Kaushik
ceff625d32 fixed formatting of function arguments for CheckFit(...) 2017-02-11 01:23:07 -05:00
Pradyumna Kaushik
13479e03a4 formatted files 2017-02-11 00:05:42 -05:00
Pradyumna Kaushik
ec78480067 renamed constants.CapMargin to constants.Tolerance for better semantics 2017-02-10 20:53:18 -05:00
Pradyumna Kaushik
f5ddc56f27 changed the name of takeOffer(...) to takeOfferBinPack(...) and then created another function called takeOfferFirstFit(...). Made sure that these functions are called instead of inlining code. 2017-02-10 16:39:13 -05:00
Pradyumna Kaushik
1dcf416849 Resolved merge conflicts with master 2017-02-10 16:23:09 -05:00