spdf/elektron
Archived
1
0
Fork 0
Code Issues 4 Pull requests 1 Projects Releases Packages Wiki Activity

Commit graph

  • 46be28ef8d pcp: CPU/MEM utilization & task share variance per node Akash Kothawale 2018-02-02 19:24:51 -05:00
  • abadb9a328 degCol: Logger for storing degree of colocation Akash Kothawale 2018-02-02 19:15:48 -05:00
  • b58ffdbdd6 formatted code Pradyumna Kaushik 2018-02-02 17:07:23 -05:00
  • 435c4ca1bc fixed bug in schedWindow resizing strategy, where earlier it was stopping after checking if the first task in the queue can be scheduled in the next offer cycle. Changed baseScheduler to store the schedWindowResizingStrategy instead of the criteria. Retrofitted the scheduling policies to use the scheduling window resizing strategy directly from baseSchedRef. Pradyumna Kaushik 2018-02-02 17:06:59 -05:00
  • 3ebd7b0c7e added utility to compute the scheduling window. Right now there's only criteria on which this is determined -- fillNextOfferCycle. So, the schedWindow is the max number of tasks, that aren't yet scheduled, whose aggregate resource requirement is as close as possible to the resource available in the next round of resource offers. To be able to make the most use of the next offer cycle, one would need to perform a non-polynomial search of the TaskQueue and as this is computationally expensive, a linear search is performed on the TaskQueue. Retrofitted scheduling policies to also call utilities.schedUtils#schedWindowResizingStrategy#Apply before switching to a new scheduling policy. Pradyumna Kaushik 2018-01-31 19:00:31 -05:00
  • 6f0f3788b9 sched: Make baseScheduler publicly accessible Akash Kothawale 2018-01-30 14:12:37 -05:00
  • 9599588fb5 formatted code. Pradyumna Kaushik 2018-01-29 18:19:44 -05:00
  • c31bf8db01 BUG Fix1: taskID creation in taskUtils#initTaskResourceRequirements. BUG Fix2: Converted value in utilities/trackResourceUsage#TrackResourceUsage#perHostResourceAvailability to be a pointer to ResourceCount. Pradyumna Kaushik 2018-01-29 18:03:46 -05:00
  • 657dc8df93 Track resource usage across the cluster. Created utility in utilities/ to track the total and the unused resources for each host in the cluster. Added utility to def/taskUtils.go to retrieve the resource requirement for a given taskID. Decoupled the code, to launch a list of tasks on a set of offerIDs, to schedulers/helpers.go and updated all the scheduling policies to call this function instead of directly calling mesos.SchedulerDriver#LaunchTasks. The resource availability of the cluster is updated at 2 stages -- 1. When the tasks are about to be launched (in schedulers/helpers.go#LaunchTasks), the scheduling policy switching logic will be able to adhere to the update in the resource availability due to the JUST launched tasks and 2. when a terminal status update is received for a task (in schedulers/base.go#statusUpdate). Pradyumna Kaushik 2018-01-26 17:29:43 -05:00
  • 6cd61ed18b comment cleanup. Pradyumna Kaushik 2018-01-20 17:26:34 -05:00
  • 3b80af6d8f Fix: mesos-go imports Akash Kothawale 2018-01-19 17:46:35 -05:00
  • 065705d480 Merged in experimentation/schedPolicySwitcher (pull request #1) Pradyumna Kaushik 2018-01-19 21:20:43 +00:00
  • cb71153362 Sorting the tasks in non-decreasing order of watts for max-greedymins and max-min. Had unintentionally removed the sorting of tasks for max-greedymins and max-mins. pradykaushik/refine refine Pradyumna Kaushik 2018-01-18 18:30:27 -05:00
  • 1b9442bf50 Fixed missing default value issue in Mesos master command line argument. Pradyumna Kaushik 2017-10-03 20:10:44 -04:00
  • 3d07efe15d removed default value for -master commandline argument. Pradyumna Kaushik 2017-10-03 01:38:34 -04:00
  • bb5df1e3ad Linked to corresponding documentation Pradyumna Kaushik 2017-09-28 15:56:16 -04:00
  • ad9f67a2a0 moved documentation of scheduling policies and power-capping strategies into a documentation folder called docs/ Pradyumna Kaushik 2017-09-28 15:53:36 -04:00
  • 4051aa1fb3 changed the named of the scheduling policy documentation to SchedulingPolicies.md. Also, added a documentation for power-capping strategies called PowerCappingStrategies.md Pradyumna Kaushik 2017-09-28 15:50:55 -04:00
  • b807625b78 Fixed the comments to be capitalized at the start and also terminate with a period. Pradyumna Kaushik 2017-09-28 15:36:47 -04:00
  • 577120ae7c removed .gitignore from the .gitignore file so as to include the gitignore file and start tracking. Pradyumna Kaushik 2017-09-28 15:14:34 -04:00
  • c2b42c2662 Now retrieving path to rapl throttle script by reading value of an environment variable. Also, stored the RAPL_PKG_Throttle.py script in a directory called scripts/. The location of this needs to be stored in an environment variable named RAPL_PKG_THROTTLE_LOCATION. Pradyumna Kaushik 2017-09-28 14:59:31 -04:00
  • dcf67661e9 removed the inlining of ssh password for username=rapl. Now we read value of environment variable named RAPL_PSSWD Pradyumna Kaushik 2017-09-27 19:52:17 -04:00
  • 971f6833c3 modified schedulers/README.md to now use Elektron instead of Electron. Also, changed the content to just display the contained scheduling policies with an explaination for each of them. Pradyumna Kaushik 2017-09-26 13:43:23 -04:00
  • 04f24beac5 scheduling policies pluggable from commandline Pradyumna Kaushik 2017-09-26 13:17:47 -04:00
  • 051aca4d10 updated pcp config file to not include stratos information Pradyumna Kaushik 2017-09-26 01:40:17 -04:00
  • 978788a5aa updated example workload definition Pradyumna Kaushik 2017-09-26 01:37:28 -04:00
  • 60e5e0ecc1 added max-min scheduling policy Pradyumna Kaushik 2017-09-26 01:14:39 -04:00
  • 3fa4f3d0e6 changed commandline argument descriptions. Changed the names of the scheduling policies and the structs. Pradyumna Kaushik 2017-09-26 01:05:18 -04:00
  • 59266d207d retrofitted to use the generic task sorter. Pradyumna Kaushik 2017-09-26 00:37:12 -04:00
  • 180e85ddfa removed unnecessary constants Pradyumna Kaushik 2017-09-26 00:31:45 -04:00
  • 2cb09ece1f fixed importing issues in power-capping strategies. Changed names of power-capping strategies to extrema.go and progressive-extrema.go Pradyumna Kaushik 2017-09-26 00:26:01 -04:00
  • 9897c983fe removed proactive-clusterwide power-capper. Retrofitted package names to use elektron and not electron Pradyumna Kaushik 2017-09-26 00:18:35 -04:00
  • 1c4b6f0f01 refined README and removed unwanted scheduling policies. pradykaushik/refineReadmeAndGitignore refineReadmeAndGitignore Pradyumna Kaushik 2017-09-26 00:05:19 -04:00
  • dc279801b7 Merged in genericTaskSorter (pull request #18) Pradyumna Kaushik 2017-09-05 18:29:13 +00:00
  • 3a9db01fa0 formatted files Pradyumna Kaushik 2017-08-26 22:33:06 -04:00
  • 8bc408dea1 fixed comments. Removed an old comment that no longer made sense. Pradyumna Kaushik 2017-08-26 15:43:09 -04:00
  • 9e620eaede changed def.TaskSorter(...) to def.SortTasks(...) for semantics. Now, def.SortTasks(...) also sorts the tasks instead of returning just a func (i, j int) bool {}. Removed TODO from the README associated with this PR Pradyumna Kaushik 2017-08-26 15:39:45 -04:00
  • 0869bea2d8 Used closures instead of strings for the generic sorter. This removed the need for reflection. Pradyumna Kaushik 2017-08-26 15:19:30 -04:00
  • 3c65bdf02e changed the scope of TaskResourceNames and SortCriteria to be package private. Pradyumna Kaushik 2017-08-23 20:10:33 -04:00
  • c2c7a2c090 Added TODO to use the generic task sorters and get rid of the existing sorters present in def/task.go Pradyumna Kaushik 2017-08-23 20:06:44 -04:00
  • f29b7f51a9 Added a generic task sorter utility. This allows for an array of tasks to be sorted based on any resource. The possible resources by which an array of tasks can be sorted currently are listed in def/sortingCriteria.go. One can add more to this if required. Pradyumna Kaushik 2017-08-23 19:35:19 -04:00
  • 9b9dc73269 Merged in kmeansTaskClassification (pull request #17) Pradyumna Kaushik 2017-08-23 02:57:33 +00:00
  • 235ed189d5 Moved the TODO, for the refinement of the cluster sorting algorithm, to the main README file. Pradyumna Kaushik 2017-08-22 13:09:05 -04:00
  • e1350838f3 Merge branch 'kmeansTaskClassification' of https://bitbucket.org/sunybingcloud/electron into kmeansTaskClassification Pradyumna Kaushik 2017-08-22 13:03:17 -04:00
  • f4459c8cbf Consolidated the ClassifyTasks(...) functions from topHeavy and bottomHeavy and added the function to def/taskUtils.go. Added TODOs for refining the means by which the kmeans classified clusters were sorted. Pradyumna Kaushik 2017-08-22 13:00:40 -04:00
  • 41ef269c62 fixed indentation Pradyumna Kaushik 2017-08-22 12:56:36 -04:00
  • e3569e95ce fixed comments to be a little more meaningful. Pradyumna Kaushik 2017-04-30 16:48:38 -04:00
  • 1b15bb14e7 Resolved merge conflicts with the master. Also, added TODO in README to use Go1.8 techniques. Pradyumna Kaushik 2017-04-21 17:17:22 -04:00
  • bb73236421 Merged in Build-bug-fix (pull request #16) ajain13 2017-03-26 01:23:32 +00:00
  • 806ac6fa86 Removed the import statement which was importing offerUtils from electron-archive. This results in build failure is electron-archive is a local package Abhishek Jain 2017-03-25 21:11:24 -04:00
  • 84c14f0c2f Merged in hostsConstantUpdation (pull request #15) ajain13 2017-03-26 00:37:41 +00:00
  • 4024f1655e Fixed a TODO Abhishek Jain 2017-03-25 20:35:50 -04:00
  • 34a9c89e45 Added a TODO regarding handling the powerclass not being configured on a node condition Abhishek Jain 2017-03-25 20:34:58 -04:00
  • de7e73ff50 Changed a log message in the UpdateEnvironment(..) func. Abhishek Jain 2017-03-25 20:27:40 -04:00
  • e01c7b1b1d Retrofitted all the schedulers to call UpdateEnvironment(..) function instead of the old named function i.e AddHostIfNew(..) Abhishek Jain 2017-03-25 18:06:39 -04:00
  • 8eddad4e17 Changed the function name from AddHostIfNew(..) to UpdateEnvironment(..) based on a comment in the PR Abhishek Jain 2017-03-25 18:05:36 -04:00
  • 85b14e125d Got rid of commented code Abhishek Jain 2017-03-24 17:28:53 -04:00
  • 9546529f80 Corrected a spelling mistake in a print statement Abhishek Jain 2017-03-24 17:13:21 -04:00
  • 1915e589cd Made a tiny change in the logging message in AddHostIfNew(..) function Abhishek Jain 2017-03-24 17:03:29 -04:00
  • ca24706582 Removed the tag TODO from a TODO Abhishek Jain 2017-03-24 16:53:27 -04:00
  • 02aa1fac3c Removed a TODO which is basically this current PR Abhishek Jain 2017-03-24 16:40:14 -04:00
  • 6665ff4776 Fixed a bug. Abhishek Jain 2017-03-24 16:36:42 -04:00
  • abc718b554 Performed a git fmt Abhishek Jain 2017-03-24 16:34:17 -04:00
  • 10358b418b Added a space before { Abhishek Jain 2017-03-24 16:31:49 -04:00
  • b4f9a989cb Retrofitted all the schedulers to now call the offerutils.AddHostIfNew(..) utility function which will dynamically populate the constants.Hosts and constants.PowerClasses. Abhishek Jain 2017-03-24 16:28:49 -04:00
  • 841c7d5ee8 Added AddHostIfNew(..) utility function which is responsible to populate the constants.Hosts and constants.PowerClasses Abhishek Jain 2017-03-24 16:27:14 -04:00
  • ed8799566b Removed the hardcoded nodes and their powerclasses from the PowerClasses map as now it will populated dynamically Abhishek Jain 2017-03-24 16:25:24 -04:00
  • ee94e13d86 Updated a TODO from the list of TODOs, changed the usage of running electron with Watts as a resource and updated the workload schema Abhishek Jain 2017-03-23 22:25:35 -04:00
  • aed4fd1073 Retrofitted all schedulers to now pick the hostname from the offer and add it to constants.Hosts Abhishek Jain 2017-03-23 22:16:05 -04:00
  • f85ed944f4 Changed the loop as now we are looping over a map and not a slice. Abhishek Jain 2017-03-23 22:13:29 -04:00
  • 710447cb0d Changed the Hosts from being a slice to a map with the key as hostname and value as a struct which would be an empty struct in order to mimic a set Abhishek Jain 2017-03-23 22:11:38 -04:00
  • 26c49d4098 Restoring offerUtils. Had deleted this by mistake. Abhishek Jain 2017-03-23 22:09:51 -04:00
  • b311ee81f5 Changed the loop as the Hosts is now a map and not a list and now looping over the keys is required. Abhishek Jain 2017-03-23 22:03:03 -04:00
  • 9365c2e51d Changed the Hosts from list to a set of hosts using a map with key as hostname and value as an empty struct Abhishek Jain 2017-03-23 22:01:29 -04:00
  • 8b7a57519b Merged in progressiveExtrema (pull request #12) Pradyumna Kaushik 2017-03-18 23:56:05 +00:00
  • dd3019735d Merge branch 'master' of https://bitbucket.org/sunybingcloud/electron into progressiveExtrema Pradyumna Kaushik 2017-03-18 19:54:37 -04:00
  • e608ce4644 Merged in newPowerClasses (pull request #14) Pradyumna Kaushik 2017-03-14 02:09:31 +00:00
  • 190b395bc3 fixed bug to return []float64{...} as observations rather than one value. Pradyumna Kaushik 2017-03-13 16:44:52 -04:00
  • 34ab753491 Used the KMeans classification to classify the tasks. Pradyumna Kaushik 2017-03-13 16:38:53 -04:00
  • 668ae586fe Merge branch 'newPowerClasses' of https://bitbucket.org/sunybingcloud/electron into kmeansTaskClassification Pradyumna Kaushik 2017-03-10 17:26:39 -05:00
  • 505ccc0471 reclassified nodes in the cluster into 4 power classes instead of the original 3. Retrofitted the hybrid schedulers TopHeavy and BottomHeavy to comply with the change classification of nodes" Pradyumna Kaushik 2017-03-10 17:03:49 -05:00
  • 41206dd82e refactored name of CapThreshold to LowerCapLimit. Added comment to mention that floating point operations can lead to precision loss. Pradyumna Kaushik 2017-03-09 19:20:13 -05:00
  • 87bd8d7cf0 Added logging when there is no victim left to cap. Pradyumna Kaushik 2017-03-09 19:17:06 -05:00
  • 1897678452 Merge branch 'master' of https://bitbucket.org/sunybingcloud/electron into kmeansTaskClassification Pradyumna Kaushik 2017-02-28 15:28:49 -05:00
  • 9eab5db2ea Merge branch 'master' of https://bitbucket.org/sunybingcloud/electron into progressiveExtrema Pradyumna Kaushik 2017-02-28 15:28:06 -05:00
  • 3ea500461d Merged in fixAggregateResourceCheckForBinPacking (pull request #13) Pradyumna Kaushik 2017-02-28 00:04:53 +00:00
  • c18dd0fa70 formatted Pradyumna Kaushik 2017-02-27 19:03:36 -05:00
  • ef8a315e90 fixed aggregate check for BinPacking. This had been broken due to a previous merge. Pradyumna Kaushik 2017-02-27 18:37:53 -05:00
  • 74eb616a72 the observation that is to be considered for the classification of a task can now be given as a function that takes the task and returns the observations as a slice of float64. Retrofitted the other functions in taskUtils to make use of this feature. Pradyumna Kaushik 2017-02-25 19:57:01 -05:00
  • e54697b0dc Added a task utility to be able to cluster tasks into N clusters based on their watts resource requirements. Electron now compatible with Go1.8 and no longer with 1.7. Added TODOs. Pradyumna Kaushik 2017-02-25 15:43:32 -05:00
  • 9dddc38cad created a utility for tasks that allows for KMeans clustering based on watts resource requirement Pradyumna Kaushik 2017-02-24 20:52:59 -05:00
  • 2939943afd Made sure that the victim that is capped the most is the one picked to be uncapped. This is to reduce the chances of benchmark starvation. Refined comments and logging. Pradyumna Kaushik 2017-02-22 20:09:04 -05:00
  • 64de61ac4e added function to initial list of Pairs from a map[string]float64. Removed unnecessary import. Pradyumna Kaushik 2017-02-22 20:07:48 -05:00
  • de4e4c0141 removed unnecessary function. Pradyumna Kaushik 2017-02-21 22:27:08 -05:00
  • 373ba63933 fixed bugs. Made sure that we don't cap a victim below the threshold. Made sure the victims the can be capped and uncapped are maintained in both cappedVictims and orderCappedVictims. Pradyumna Kaushik 2017-02-21 21:05:47 -05:00
  • 531c1a120b Fixed messages for commandline arguments. Pradyumna Kaushik 2017-02-20 23:49:27 -05:00
  • 7d93215a7c added TODO for a generic task sorter that sorts based on any kind of resource, instead of having a sorter for each kind of resource. Pradyumna Kaushik 2017-02-20 22:55:56 -05:00
  • ef6f74cd81 removed TODO for consolidating common scheduler struct members into base.go. Pradyumna Kaushik 2017-02-20 22:42:07 -05:00
  • 726c0555ed Fixed corner cases in progressive extrema -- When a node is capped and the new cap value is above a threshold then that node can be capped or uncapped in the next cycle. If the new cap value is equal to the threshold then the node cannot be capped further and can only be uncapped. When the node is uncapped and the newUncapValue is below 100 then the node can be capped or uncapped in the next cycle. If the newUncapValue is 100 then the node can only be capped. Pradyumna Kaushik 2017-02-20 20:55:06 -05:00
  • d42b7a3a3b changed the type of percentage in rapl.Cap(...) from int to float64. Retrofitted power-capping strategies to cap using a float64 value instead of an int. Moved common functions in loganddynamiccap.go and logAndProgressiveExtrema.go into pcp/utils.go. New power-capping strategy that builds on top of extrema, where it caps the victims at different until it can't cap further, in which case it starts uncapping them in the reverse order of capping. Pradyumna Kaushik 2017-02-15 19:22:56 -05:00