Renan DelValle
45f9efa578
Changing imports from bitbucket to gitlab.
2018-09-30 17:56:14 -07:00
Pradyumna Kaushik
5a28f8539a
formatted code.
2018-09-23 18:23:37 -04:00
Akash Kothawale
acf9332a50
Comments: Explain what Variance is used for.
2018-09-23 18:16:27 -04:00
Akash Kothawale
2d3158d99a
pcp: Remove unused functions
2018-09-23 18:16:17 -04:00
Akash Kothawale
c52bfb08c5
Handle case: host -> slaveID may not be available
2018-09-23 18:16:06 -04:00
Akash Kothawale
9c09321d81
Fix incorrect degColLog file location
2018-09-23 18:14:48 -04:00
Akash Kothawale
39d1cf980c
Use existing Mean and Variance functions
...
Available in github.com/montanaflynn/stats
2018-09-23 18:13:47 -04:00
Akash Kothawale
46be28ef8d
pcp: CPU/MEM utilization & task share variance per node
2018-09-23 18:13:28 -04:00
Pradyumna Kaushik
065705d480
Merged in experimentation/schedPolicySwitcher (pull request #1 )
...
Experimentation/schedPolicySwitcher
1. Initial commit for consolidated loggers using observer pattern.
2. class factory for schedulers.
3. Using the scheduling policy class factory in schedulers/store.go and the scheduler builder helpers in schedulers/helpers.go, feature to be able to be able to plug a scheduling policy of your choice from the command line (right now only first-fit and bin-packing are possible. Will be updating the class factory to include other scheduling policies as well.
4. Removed TODO for using generic task sorters. Modified TODO for a config file input to run electron.
5. Added other schedulers to the factory
6. Partially retrofitted the other scheduling policies to use the logging library.
7. Retrofitted extrema and progressive to use the consolidated logging library. Fixed parameter issue with s.base.Disconnected(). Formatted project
8. Move statusUpdate(...) into base.go to remove redundant code.
9. Converted the baseScheduler into a state machine where the state is a scheduling policy that defines an approach to consume resource offers.
10. Added another command line argument to be used to enable switching of scheduling policies. Retrofitted scheduling policies to switch only if the particular feature has been enabled.
changed argument to coLocated(...) to take base type rather than ElectronScheduler type. Also, prepended the prefix to the directory of the logs so that it would be easier to determine what the files in a directory correspond to without viewing the contents of the directory.
Defined methods in ElectronScheduler. Each of these methods corresponds to a type of log that an ElectronScheduler would make. Each of these methods would need to be implemented by the scheduling policy.
Electron has only one scheduler that implements the mesos scheduler interface. All the scheduling policies are just different implementations of ways to consume mesos resource offers. Retrofitted scheduling policies to now embed SchedPolicyState instead of baseScheduler.
Approved-by: Pradyumna Kaushik <pkaushi1@binghamton.edu>
2018-09-23 17:19:28 -04:00
Pradyumna Kaushik
b807625b78
Fixed the comments to be capitalized at the start and also terminate with a period.
2017-09-28 15:36:47 -04:00
Pradyumna Kaushik
2cb09ece1f
fixed importing issues in power-capping strategies. Changed names of power-capping strategies to extrema.go and progressive-extrema.go
2017-09-26 00:26:01 -04:00
Pradyumna Kaushik
9897c983fe
removed proactive-clusterwide power-capper. Retrofitted package names to use elektron and not electron
2017-09-26 00:18:35 -04:00
Abhishek Jain
85b14e125d
Got rid of commented code
2017-03-24 17:28:53 -04:00
Abhishek Jain
f85ed944f4
Changed the loop as now we are looping over a map and not a slice.
2017-03-23 22:13:29 -04:00
Pradyumna Kaushik
41206dd82e
refactored name of CapThreshold to LowerCapLimit. Added comment to mention that floating point operations can lead to precision loss.
2017-03-09 19:20:13 -05:00
Pradyumna Kaushik
87bd8d7cf0
Added logging when there is no victim left to cap.
2017-03-09 19:17:06 -05:00
Pradyumna Kaushik
2939943afd
Made sure that the victim that is capped the most is the one picked to be uncapped. This is to reduce the chances of benchmark starvation. Refined comments and logging.
2017-02-22 20:09:04 -05:00
Pradyumna Kaushik
373ba63933
fixed bugs. Made sure that we don't cap a victim below the threshold. Made sure the victims the can be capped and uncapped are maintained in both cappedVictims and orderCappedVictims.
2017-02-21 21:05:47 -05:00
Pradyumna Kaushik
726c0555ed
Fixed corner cases in progressive extrema -- When a node is capped and the new cap value is above a threshold then that node can be capped or uncapped in the next cycle. If the new cap value is equal to the threshold then the node cannot be capped further and can only be uncapped. When the node is uncapped and the newUncapValue is below 100 then the node can be capped or uncapped in the next cycle. If the newUncapValue is 100 then the node can only be capped.
2017-02-20 20:55:06 -05:00
Pradyumna Kaushik
d42b7a3a3b
changed the type of percentage in rapl.Cap(...) from int to float64. Retrofitted power-capping strategies to cap using a float64 value instead of an int. Moved common functions in loganddynamiccap.go and logAndProgressiveExtrema.go into pcp/utils.go. New power-capping strategy that builds on top of extrema, where it caps the victims at different until it can't cap further, in which case it starts uncapping them in the reverse order of capping.
2017-02-15 19:22:56 -05:00
Pradyumna Kaushik
a0a3e78041
Merged in hybridSchedulerAndSortedOffers (pull request #8 )
...
Hybrid Scheduler -- TopHeavy, BottomHeavy, FirstFit and BinPacked schedulers with sortedOffers.
2017-02-10 20:21:04 +00:00
Renan DelValle
12efaedaa9
Style and naming fixes.
2017-02-05 14:54:15 -05:00
Renan DelValle
610c006834
Adding DRAM monitoring to Extrema.
2017-02-03 23:12:49 -05:00
Pradyumna Kaushik
0413d2abd3
moved proactiveclusterwidecappers.go to powerCapping/ from pcp/
2017-01-28 21:42:23 -05:00
Pradyumna Kaushik
2cd77a7ba8
Changed Window to ConsiderationWindow.
2017-01-28 18:29:00 -05:00
Pradyumna Kaushik
8c6ad36b5e
changed name of StartLogAndDynamicCap(...) to StartPCPLogAndExtremaDynamicCap(...).]
2017-01-15 15:23:57 -05:00
Pradyumna Kaushik
8c0cde939a
renamed Recap to NaiveRecap
2017-01-08 19:13:44 -08:00
Renan DelValle
2678032c2c
Go fmt on the entire project
2017-01-03 20:57:25 -05:00
Renan DelValle
b73c30a8bf
Implemented a scheduling tracer. Logs time date and host:task-ID. Made log prefix more global since it needs to be constant for all log files generated.
2017-01-03 20:30:21 -05:00
Renan DelValle
147bbc8b5a
Fixing commit history
2016-12-22 23:27:18 -05:00
Pradyumna Kaushik
f829cfea66
Moved it to pcp/ as it is not a scheduler and is a capping strategy
2016-12-22 23:20:18 -05:00
Pradyumna Kaushik
79a5dca76b
Added another log message to log the name of the file to which the pcplogs are getting written to.
2016-12-22 22:58:59 -05:00
Pradyumna Kaushik
383f088b6a
changed bingcloud to sunybingcloud in the import statements.
2016-12-22 22:58:59 -05:00
Renan DelValle
d2fc72885d
Go FMT run on entire project
2016-12-22 22:58:59 -05:00
Renan DelValle
bf9a5a6005
Finished integrating dynamic capping policy that uses a high and a low threshold to start capping and uncapping node via an ssh commands that triggers a python script. Commit has sensitive data, scrub before releasing to public.
2016-12-22 22:58:59 -05:00
Renan DelValle
f7e38285a4
rapl capping via ssh command in rapl package
2016-12-22 22:58:59 -05:00
Renan DelValle
fbc05a42df
Initial version of command which will cap system
2016-12-22 22:58:59 -05:00
Renan DelValle
7d57957524
Averages are calculated correctly, PCP is able to determine a victim to be capped
2016-12-22 22:58:59 -05:00
Renan DelValle
fce62981da
Moved schedulers from the main programs to schedulers package. Can now choose different scheduelrs to use. Work on code sharing between schedulers remains to be done.
2016-12-22 22:58:58 -05:00
Renan DelValle
6a406cad69
SIGINT is now captured. Linux/MacOS shutdown for pcpdumptext child
...
processes added to clean up after we finish running. Made all tasks
run on bridged network mode so that benchmarks that heavily utilize
network are protected from eachother. I.e: Tradebeans, tradesoap
2016-12-22 22:58:58 -05:00
Renan DelValle
086b06256d
Added Error state. Fixed bug with tasks returning error because 0.0 watts was requested as a resource. Changed name to be more readable by eliminating electron. PCP logs can now have a prefix.
2016-12-22 22:58:58 -05:00
Renan DelValle
2575c2a20b
PCP is now controlled by main scheduler. It will start recording upon the acceptance of the very first offer. pcp is now it's own package. README has been updated with instructions on how to create workloads and running instructions.
2016-12-22 22:58:58 -05:00
Renan DelValle
1b57b6dec2
Tied metric columns to values, next step will be determining how to store them and retrieve them
2016-12-22 22:58:58 -05:00
Renan DelValle
fa3a620fba
PCP code is now able to deal with receiving information asynchronously from pmdumptext
2016-12-22 22:58:58 -05:00
