Commit graph

91 commits

Author SHA1 Message Date
balandi1
027adef642 Code and formatting fixes
Changed the shorthand for log config flag.
Fixed formatting as mentioned.
Removed unnecessary function declarations from elektronLogger interface.
Disabled schedWindow and clsfnTaskDistrOverhead in config file.
2019-12-09 10:55:34 -05:00
Pradyumna Kaushik
12f827c0fe Used WithField() and Logf() where required.
Used elektronLogger#WithField(...) for single key-value pair.
Used elektronLogger#Logf(...) where formatted string is required.
2019-12-05 22:33:28 -05:00
Pradyumna Kaushik
3b70a13cc8 refactored logging/types imports to dot imports 2019-12-05 21:43:16 -05:00
Pradyumna Kaushik
6fb0e4a3fe move cfg to loggers + refactor + log fn wrappers
1. Instead of maintaining a global config, each specialized logger
now stores its config.
2. Refactored logInterface to elektronLogger.
3. Refactored loggerImpl to baseElektronLogger to be consistent with
the rest of the code base.
4. Wrapped elektronLogger#Log(...) and elektronLogf(...) so that we
do not have to use the instance of elektronLogger everytime we want
to log. Instead, we just do logging.Log(...) or logging.Logf(...).
5. Wrapped elektronLogger#WithFields(...) and
elektronLogger#WithField(...).
6. Refactored codebase to adhere to the changes.
2019-12-05 21:32:37 -05:00
Pradyumna Kaushik
3d201bf437 refactored elektronLogging to logging 2019-12-05 17:38:56 -05:00
balandi1
bedfa52d7a Added functions to logging library
Added Logf() and WithFields() functions.
Logf() for logging formatted messages.
WithFields() for handling optional log fields.
2019-12-04 13:16:48 -05:00
balandi1
0c0361e209 Code Refactoring
Modified comments. Organized imports.
2019-11-26 14:24:20 -05:00
balandi1
24dbbb208f Flag for Logger Config yaml file name
Removed yaml filename from env.go and added its flag to scheduler.go
2019-11-26 13:08:45 -05:00
balandi1
cb5df9acc8 Removed ERROR, GENERAL, SUCCESS, WARNING log types 2019-11-21 14:58:14 -05:00
balandi1
268df3cd51 Changed alias for elektronLogging to elekLog 2019-11-21 14:35:20 -05:00
balandi1
270c8669e6 Revert previous commit. Renamed wrong alias 2019-11-21 14:12:53 -05:00
balandi1
9952b9861d Renamed alias log to elekLog 2019-11-21 14:02:47 -05:00
balandi1
13c3de44be Changed alias elekLogT to elekLogTypes 2019-11-21 13:05:52 -05:00
balandi1
20f2f1e8c5 Used string join at neccesary places. Removed unused code 2019-11-20 16:11:33 -05:00
balandi1
4d15e59d4c Code formatting 2019-11-20 13:33:46 -05:00
balandi1
c27aba895b Changes made everywhere inorder to use elektronLogging library for logging 2019-11-13 14:08:52 -05:00
PRADYUMNA KAUSHIK
e3caa90c31
Refactor imports, Add vendor/ as submodule (#5)
refact imports to github.xxx. update go.mod + go.sum

* removed vendor/ folder

* Added vendor/ submodule.

Moved dependencies that need to be vendored into separate repository.
	- https://github.com/spdfg/elektron-vendor
Added vendor/ as a submodule using the below command.
	- git submodule add https://github.com/spdfg/elektron-vendor vendor
If wanting to use vendor, run the following commands after cloning
elektron.
1. git submodule init
2. git submodule update

* added instructions to clone vendor/ submodule.

* updated module to spdfg. Refactored imports
2019-10-31 14:32:46 -04:00
Pradyumna Kaushik
4fbbf91b00 Switch to go modules for dependency management.
Added support for dependency management using go modules.
Note that Go version 1.11+ is required if using go modules for
dependency management.

Updated the gitlab CI yaml file to use go modules instead of copying
everything to GOPATH. Ran go mod download before running build script to
download all dependencies into cache.
2019-10-02 00:21:21 +00:00
Pradyumna Kaushik
cfbb2f4bdd Fix: create log dir after validating cmdline args.
Initialization of the logger happens after all the command-line
arguments are validated. So, in case any of the command-line
arguments were invalid, the log directory is not created.
Retrofitted the driver code and the scheduler builder to only use
the language provided 'log' package as elektron's logger will not
yet be initialized.
2018-11-08 20:07:06 -05:00
Renan DelValle
291f4f3979
Adding GPLv3 Licensing. 2018-10-06 20:03:14 -07:00
Pradyumna Kaushik
bcc2b4e4cc Removed Degree of Collocation logger. 2018-10-05 14:11:49 -04:00
Pradyumna Kaushik
6f97c36411 Passed scheduler to extrema and prog-extrema. 2018-10-05 00:41:43 -04:00
Pradyumna Kaushik
5a6a43937c Hi/Lo Thresholds only for extrema and prog-extrema.
Added condition to mandate high and low thresholds only for extrema
and progressive extrema power capping strategies.
2018-10-04 20:10:46 -04:00
Pradyumna Kaushik
4637355721 Reorganized code.
Building the scheduler in phases. To be able to do this, increased
visibility of schedulers/helpers.go#schedulerOptions.
Refactored dependent code.
2018-10-04 19:27:13 -04:00
Pradyumna Kaushik
c9d4e66236 Powercap policy and PCP config from commandline.
Added command line arguments corresponding to powercapping policy
and pcp config file path.
2018-10-04 19:21:45 -04:00
Pradyumna Kaushik
1e2fae9f80 Removed underscore for second return. 2018-10-04 14:11:38 -04:00
Pradyumna Kaushik
1e9828c35b Provide pcp config file from commandline.
Taking pcp config file name (or path) from commandline.
In scheduler.go, also checking to see if provided pcp config file
exists. If not, then exitting.
2018-10-04 13:57:35 -04:00
Pradyumna Kaushik
8e87bcb439 Fixed import alias prefix.
Changed the prefix to import aliases to be 'elek' instead of 'elec'.
2018-10-04 13:45:31 -04:00
Pradyumna Kaushik
b06bdeba59 Used println instead of printf if no args. 2018-10-04 13:40:04 -04:00
Pradyumna Kaushik
49b58fd2f9 Plug-in power-capping policy from commandline. 2018-10-04 13:38:55 -04:00
Renan DelValle
9554bf2666
Runing goimports on project as well as fixing README so that headers render correctly. 2018-09-30 18:23:38 -07:00
Renan DelValle
45f9efa578
Changing imports from bitbucket to gitlab. 2018-09-30 17:56:14 -07:00
Pradyumna Kaushik
66c19b53c9 Merged in differentSwitchingMechanisms (pull request #14)
DifferentSwitchingMechanisms

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
2018-09-23 19:21:52 -04:00
Pradyumna Kaushik
8b27f5547d Merged in fixSchedWindowSize (pull request #13)
FixSchedWindowSize

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
2018-09-23 19:18:50 -04:00
Pradyumna Kaushik
ae81125110 Merged in mapTaskDistrToSchedPolWhenSwitching (pull request #11)
MapTaskDistrToSchedPolWhenSwitching

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
2018-09-23 19:18:07 -04:00
Pradyumna Kaushik
b877d31cb8 Merged in schedPolSwitchConfigFile (pull request #8)
SchedPolSwitchConfigFile

Approved-by: Akash Kothawale <akothaw1@binghamton.edu>
2018-09-23 19:10:23 -04:00
Akash Kothawale
46be28ef8d pcp: CPU/MEM utilization & task share variance per node 2018-09-23 18:13:28 -04:00
Pradyumna Kaushik
3ebd7b0c7e added utility to compute the scheduling window. Right now there's only criteria on which this is determined -- fillNextOfferCycle. So, the schedWindow is the max number of tasks, that aren't yet scheduled, whose aggregate resource requirement is as close as possible to the resource available in the next round of resource offers. To be able to make the most use of the next offer cycle, one would need to perform a non-polynomial search of the TaskQueue and as this is computationally expensive, a linear search is performed on the TaskQueue. Retrofitted scheduling policies to also call utilities.schedUtils#schedWindowResizingStrategy#Apply before switching to a new scheduling policy. 2018-09-23 18:03:14 -04:00
Akash Kothawale
3b80af6d8f Fix: mesos-go imports 2018-09-23 17:31:03 -04:00
Pradyumna Kaushik
065705d480 Merged in experimentation/schedPolicySwitcher (pull request #1)
Experimentation/schedPolicySwitcher
1. Initial commit for consolidated loggers using observer pattern.
2. class factory for schedulers.
3. Using the scheduling policy class factory in schedulers/store.go and the scheduler builder helpers in schedulers/helpers.go, feature to be able to be able to plug a scheduling policy of your choice from the command line (right now only first-fit and bin-packing are possible. Will be updating the class factory to include other scheduling policies as well.
4. Removed TODO for using generic task sorters. Modified TODO for a config file input to run electron.
5. Added other schedulers to the factory
6. Partially retrofitted the other scheduling policies to use the logging library.
7. Retrofitted extrema and progressive to use the consolidated logging library. Fixed parameter issue with s.base.Disconnected(). Formatted project
8. Move statusUpdate(...) into base.go to remove redundant code.
9. Converted the baseScheduler into a state machine where the state is a scheduling policy that defines an approach to consume resource offers.
10. Added another command line argument to be used to enable switching of scheduling policies. Retrofitted scheduling policies to switch only if the particular feature has been enabled.

changed argument to coLocated(...) to take base type rather than ElectronScheduler type. Also, prepended the prefix to the directory of the logs so that it would be easier to determine what the files in a directory correspond to without viewing the contents of the directory.
Defined methods in ElectronScheduler. Each of these methods corresponds to a type of log that an ElectronScheduler would make. Each of these methods would need to be implemented by the scheduling policy.

Electron has only one scheduler that implements the mesos scheduler interface. All the scheduling policies are just different implementations of ways to consume mesos resource offers. Retrofitted scheduling policies to now embed SchedPolicyState instead of baseScheduler.

Approved-by: Pradyumna Kaushik <pkaushi1@binghamton.edu>
2018-09-23 17:19:28 -04:00
Pradyumna Kaushik
1b9442bf50 Fixed missing default value issue in Mesos master command line argument. 2017-10-03 20:10:44 -04:00
Pradyumna Kaushik
3d07efe15d removed default value for -master commandline argument. 2017-10-03 01:38:34 -04:00
Pradyumna Kaushik
b807625b78 Fixed the comments to be capitalized at the start and also terminate with a period. 2017-09-28 15:36:47 -04:00
Pradyumna Kaushik
04f24beac5 scheduling policies pluggable from commandline 2017-09-26 13:17:47 -04:00
Pradyumna Kaushik
3fa4f3d0e6 changed commandline argument descriptions. Changed the names of the scheduling policies and the structs. 2017-09-26 01:05:18 -04:00
Pradyumna Kaushik
9897c983fe removed proactive-clusterwide power-capper. Retrofitted package names to use elektron and not electron 2017-09-26 00:18:35 -04:00
Pradyumna Kaushik
41ef269c62 fixed indentation 2017-08-22 12:56:36 -04:00
Abhishek Jain
9546529f80 Corrected a spelling mistake in a print statement 2017-03-24 17:13:21 -04:00
Pradyumna Kaushik
531c1a120b Fixed messages for commandline arguments. 2017-02-20 23:49:27 -05:00
Pradyumna Kaushik
726c0555ed Fixed corner cases in progressive extrema -- When a node is capped and the new cap value is above a threshold then that node can be capped or uncapped in the next cycle. If the new cap value is equal to the threshold then the node cannot be capped further and can only be uncapped. When the node is uncapped and the newUncapValue is below 100 then the node can be capped or uncapped in the next cycle. If the newUncapValue is 100 then the node can only be capped. 2017-02-20 20:55:06 -05:00