spdf/elektron
Archived
1
0
Fork 0
Code Issues 4 Pull requests 1 Projects Releases Packages Wiki Activity

Elektron Logging library (#16)

Browse source 
switch to logrus for logging.

replaced old logging library with a wrapper around logrus.
We now just need to use the exported Log(...) and Logf(...) from the logging/
package that wraps around a set of loggers constituting a chain (following COR).
Loggers are configured using a YAML file that specifies the following.
1. enabled/disabled
2. whether the message should be logged on console.
3. filename extension.
4. minimum log level.

Retrofitted source code to now use the updated logging library.
Updated the documentation with information regarding the specification
of the log config file.

Currently, the log format in the config file is not adhered to. This is going to be
addressed in a future commit.
This commit is contained in:
Bhargavi Hanumant Alandikar 2019-12-09 20:15:33 -05:00 • committed by PRADYUMNA KAUSHIK
parent 9977251c14
commit 3543960689
61 changed files with 1510 additions and 1067 deletions
Download patch file Download diff file Expand all files Collapse all files

10
constants/constants.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// any later version. // any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
// TODO: Clean this up and use Mesos Attributes instead. // TODO: Clean this up and use Mesos Attributes instead.
package constants package constants

10
def/metric.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package def package def

10
def/sortingCriteria.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package def package def

10
def/task.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package def package def

18
def/taskUtils.go
View file

@ -1,31 +1,33 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package def package def
import ( import (
"errors" "errors"
"fmt" "fmt"
"log"
"sort" "sort"
"github.com/mash/gokmeans" "github.com/mash/gokmeans"
"github.com/montanaflynn/stats" "github.com/montanaflynn/stats"
log "github.com/sirupsen/logrus"
elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
) )
// Information about a cluster of tasks. // Information about a cluster of tasks.
@ -50,7 +52,7 @@ func (tc TasksToClassify) taskObservationCalculator(task Task) []float64 {
} else if task.Watts != 0.0 { } else if task.Watts != 0.0 {
return []float64{task.Watts} return []float64{task.Watts}
} else { } else {
log.Fatal("Unable to classify tasks. Missing Watts or ClassToWatts attribute in workload.") elekLog.Log(CONSOLE, log.FatalLevel, "Unable to classify tasks. Missing Watts or ClassToWatts attribute in workload")
return []float64{0.0} // Won't reach here. return []float64{0.0} // Won't reach here.
} }
} }
@ -105,7 +107,7 @@ func clusterSizeAvgMMMPU(tasks []Task, taskObservation func(task Task) []float64
} else { } else {
// skip this value // skip this value
// there is an error in the task config. // there is an error in the task config.
log.Println(err) elekLog.Log(CONSOLE, log.ErrorLevel, err.Error())
} }
} else { } else {
// There is only one observation for the task. // There is only one observation for the task.

5
def/task_test.go
View file

@ -20,11 +20,12 @@ package def
import ( import (
"fmt" "fmt"
"reflect"
"testing"
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
"github.com/spdfg/elektron/constants" "github.com/spdfg/elektron/constants"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"reflect"
"testing"
) )
func TestTasksFromJSON(t *testing.T) { func TestTasksFromJSON(t *testing.T) {

16
docs/Logs.md
View file

@ -11,4 +11,18 @@ _Elektron_ logs can be categorized into the following.
* _**Scheduling Policy Switching Logs**_ - When scheduling policy switching is enabled (`-switchSchedPol` is used when launching _Elektron_), the following information is logged. * _**Scheduling Policy Switching Logs**_ - When scheduling policy switching is enabled (`-switchSchedPol` is used when launching _Elektron_), the following information is logged.
* [**Scheduling Policy Switch trace (SPS)**](data/withSpsEnabled/SchedulingPolicySwitchTrace.md) - Every time _Elektron_ switches to a different scheduling policy, the _name_ of the scheduling policy and the corresponding _time stamp_ is logged.<br> * [**Scheduling Policy Switch trace (SPS)**](data/withSpsEnabled/SchedulingPolicySwitchTrace.md) - Every time _Elektron_ switches to a different scheduling policy, the _name_ of the scheduling policy and the corresponding _time stamp_ is logged.<br>
* [**SCHED_WINDOW**](data/withSpsEnabled/SchedulingWindow.md) - For every switch, the size of the scheduling window and the name of the scheduling policy is logged. * [**SCHED_WINDOW**](data/withSpsEnabled/SchedulingWindow.md) - For every switch, the size of the scheduling window and the name of the scheduling policy is logged.
* [**CLSFN_TASKDIST_OVERHEAD**](data/withSpsEnabled/TaskClassificationOverhead.md) - If the switching criteria is task distribution based, then the time taken to classify the tasks into low power consuming and high power consuming, and then to determine the task distribution is logged. * [**CLSFN_TASKDISTR_OVERHEAD**](data/withSpsEnabled/TaskClassificationOverhead.md) - If the switching criteria is task distribution based, then the time taken to classify the tasks into low power consuming and high power consuming, and then to determine the task distribution is logged.
_Elektron_ logs can be configured through [Log config file](../logConfig.yaml). The following is the format for configuration.
```
<logtype>:
enabled: <true/false>
filenameExtension: <filename extension>
minLogLevel: <minimum log level>
allowOnConsole: <true/false>
```
The file has default configurations set. One can also configure the above fields for every log type.
* `enabled` - Enable or disable a specific log type.
* `filenameExtension` - Provide the file extension for specific log type.
* `minLogLevel` - Provide a minimum log level above which all logs should be logged. This is available only for Console log type. The default value is debug).
* `allowOnConsole` - Allow or Disallow a specific log type on the console.

12
docs/data/PCP.md
View file

@ -25,12 +25,12 @@ myhost:kernel.all.cpu.sys
When we run the `pmdumptext` command mentioned above for 5 seconds, the PCP log from _Elektron_ would be as shown below. When we run the `pmdumptext` command mentioned above for 5 seconds, the PCP log from _Elektron_ would be as shown below.
``` ```
<yyyy/mm/dd> <hh:mm:ss> myhost:kernel.all.cpu.user,myhost:kernel.all.cpu.sys [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> myhost:kernel.all.cpu.user,myhost:kernel.all.cpu.sys
<yyyy/mm/dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time> [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time>
<yyyy/mm/dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time> [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time>
<yyyy/mm/dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time> [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time>
<yyyy/mm/dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time> [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time>
<yyyy/mm/dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time> [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> <myhost user cpu time>,<myhost system cpu time>
``` ```
## Power Measurements ## Power Measurements

2
docs/data/ScheduledTrace.md
View file

@ -9,6 +9,6 @@ The scheduled trace logs are written to a file named _\<logFilePrefix\>\_\<times
The format of the data logged is as shown below. The format of the data logged is as shown below.
``` ```
<yyyy/mm/dd> <hh:mm:ss> <hostname>:<task ID> [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> <hostname>=<task ID>
``` ```

2
docs/data/withSpsEnabled/SchedulingPolicySwitchTrace.md
View file

@ -8,7 +8,7 @@ The logs are written to a file named _\<logFilePrefix\>\_\<timestamp\>\_schedPol
The format of the data logged is as shown below. The format of the data logged is as shown below.
``` ```
<yyyy/mm/dd> <hh:mm:ss> <sched policy name> [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> Name=<sched policy name>
``` ```
_Note: The names of the scheduling policies can be found [here](https://gitlab.com/spdf/elektron/blob/master/schedulers/store.go#L14)_ _Note: The names of the scheduling policies can be found [here](https://gitlab.com/spdf/elektron/blob/master/schedulers/store.go#L14)_

4
docs/data/withSpsEnabled/SchedulingWindow.md
View file

@ -8,5 +8,5 @@ The logs are written to a file named _\<logFilePrefix\>\_\<timestamp\>\_schedWin
The format of the data logged is as shown below. The format of the data logged is as shown below.
``` ```
<yyyy/mm/dd> <hh:mm:ss> <sched window size> <sched policy name> [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> Window size=<sched window size>, Name=<sched policy name>
``` ```

2
docs/data/withSpsEnabled/TaskClassificationOverhead.md
View file

@ -8,7 +8,7 @@ The logs are written to a file named _\<logFilePrefix\>\_\<timestamp\>\_classifi
The format of the data logged is as shown below. The format of the data logged is as shown below.
``` ```
<yyyy/mm/dd> <hh:mm:ss> <overhead> [<loglevel>]: <yyyy-mm-dd> <hh:mm:ss> Overhead in microseconds=<overhead>
``` ```
_Note: The classification overhead is logged in **microseconds**._ _Note: The classification overhead is logged in **microseconds**._

10
environment/env.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package environment package environment

29
go.mod
View file

@ -4,20 +4,23 @@ go 1.12
require ( require (
github.com/fatih/color v1.7.0 github.com/fatih/color v1.7.0
github.com/gogo/protobuf v1.1.1 github.com/gogo/protobuf v1.3.0
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/golang/protobuf v1.2.0 github.com/golang/protobuf v1.3.2
github.com/google/uuid v1.0.0 github.com/google/uuid v1.0.0
github.com/mash/gokmeans v0.0.0-20140614041449-8bbf08905a7e github.com/mash/gokmeans v0.0.0-20170215130432-ea22cff45f59
github.com/mattn/go-colorable v0.0.9 github.com/mattn/go-colorable v0.1.2
github.com/mattn/go-isatty v0.0.4 github.com/mattn/go-isatty v0.0.9
github.com/mesos/mesos-go v0.0.8 github.com/mesos/mesos-go v0.0.10
github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe github.com/montanaflynn/stats v0.5.0
github.com/pborman/uuid v0.0.0-20180906182336-adf5a7427709 github.com/pborman/uuid v1.2.0
github.com/pkg/errors v0.8.0 github.com/pkg/errors v0.8.1
github.com/samuel/go-zookeeper v0.0.0-20180130194729-c4fab1ac1bec github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da
github.com/sirupsen/logrus v1.4.2
github.com/stretchr/testify v1.4.0 github.com/stretchr/testify v1.4.0
golang.org/x/crypto v0.0.0-20180927165925-5295e8364332 gitlab.com/spdf/elektron v0.0.0-20191024200717-26f96f361f10
golang.org/x/net v0.0.0-20180926154720-4dfa2610cdf3 golang.org/x/crypto v0.0.0-20190927123631-a832865fa7ad
golang.org/x/sys v0.0.0-20180928133829-e4b3c5e90611 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3
golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a
gopkg.in/yaml.v2 v2.2.2
) )

44
go.sum
View file

@ -1,41 +1,85 @@
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8= github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys= github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/gogo/protobuf v1.1.1 h1:72R+M5VuhED/KujmZVcIquuo8mBgX4oVda//DQb3PXo= github.com/gogo/protobuf v1.1.1 h1:72R+M5VuhED/KujmZVcIquuo8mBgX4oVda//DQb3PXo=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.3.0 h1:G8O7TerXerS4F6sx9OV7/nRfJdnXgHZu/S/7F2SN+UE=
github.com/gogo/protobuf v1.3.0/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM= github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/google/uuid v1.0.0 h1:b4Gk+7WdP/d3HZH8EJsZpvV7EtDOgaZLtnaNGIu1adA= github.com/google/uuid v1.0.0 h1:b4Gk+7WdP/d3HZH8EJsZpvV7EtDOgaZLtnaNGIu1adA=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/mash/gokmeans v0.0.0-20140614041449-8bbf08905a7e h1:KrPVKjYg/ocMUbv39AQoIINny7acoDE0jghg+vTvTY4= github.com/mash/gokmeans v0.0.0-20140614041449-8bbf08905a7e h1:KrPVKjYg/ocMUbv39AQoIINny7acoDE0jghg+vTvTY4=
github.com/mash/gokmeans v0.0.0-20140614041449-8bbf08905a7e/go.mod h1:nxv+mR7KuDZRAmEjsCFBQNtWcIty99qUn2Ycbgcns5c= github.com/mash/gokmeans v0.0.0-20140614041449-8bbf08905a7e/go.mod h1:nxv+mR7KuDZRAmEjsCFBQNtWcIty99qUn2Ycbgcns5c=
github.com/mash/gokmeans v0.0.0-20170215130432-ea22cff45f59 h1:C7rqeTVPDC/0QZFxfsMvS2y66WjMxQogK8/PTy9Jo0A=
github.com/mash/gokmeans v0.0.0-20170215130432-ea22cff45f59/go.mod h1:nxv+mR7KuDZRAmEjsCFBQNtWcIty99qUn2Ycbgcns5c=
github.com/mattn/go-colorable v0.0.9 h1:UVL0vNpWh04HeJXV0KLcaT7r06gOH2l4OW6ddYRUIY4= github.com/mattn/go-colorable v0.0.9 h1:UVL0vNpWh04HeJXV0KLcaT7r06gOH2l4OW6ddYRUIY4=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-colorable v0.1.2 h1:/bC9yWikZXAL9uJdulbSfyVNIR3n3trXl+v8+1sx8mU=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.4 h1:bnP0vzxcAdeI1zdubAl5PjU6zsERjGZb7raWodagDYs= github.com/mattn/go-isatty v0.0.4 h1:bnP0vzxcAdeI1zdubAl5PjU6zsERjGZb7raWodagDYs=
github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.9 h1:d5US/mDsogSGW37IV293h//ZFaeajb69h+EHFsv2xGg=
github.com/mattn/go-isatty v0.0.9/go.mod h1:YNRxwqDuOph6SZLI9vUUz6OYw3QyUt7WiY2yME+cCiQ=
github.com/mesos/mesos-go v0.0.8 h1:hiAUHba+ycyZLxDiBUqKs91a0LbHZaAca988LzN19xM= github.com/mesos/mesos-go v0.0.8 h1:hiAUHba+ycyZLxDiBUqKs91a0LbHZaAca988LzN19xM=
github.com/mesos/mesos-go v0.0.8/go.mod h1:kPYCMQ9gsOXVAle1OsoY4I1+9kPu8GHkf88aV59fDr4= github.com/mesos/mesos-go v0.0.8/go.mod h1:kPYCMQ9gsOXVAle1OsoY4I1+9kPu8GHkf88aV59fDr4=
github.com/mesos/mesos-go v0.0.10 h1:+M/7Zlkvw4MolkLvXHfj6hkDsLLHOOU54CmOkOUaNBc=
github.com/mesos/mesos-go v0.0.10/go.mod h1:kPYCMQ9gsOXVAle1OsoY4I1+9kPu8GHkf88aV59fDr4=
github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe h1:iruDEfMl2E6fbMZ9s0scYfZQ84/6SPL6zC8ACM2oIL0= github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe h1:iruDEfMl2E6fbMZ9s0scYfZQ84/6SPL6zC8ACM2oIL0=
github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe/go.mod h1:wL8QJuTMNUDYhXwkmfOly8iTdp5TEcJFWZD2D7SIkUc= github.com/montanaflynn/stats v0.0.0-20171201202039-1bf9dbcd8cbe/go.mod h1:wL8QJuTMNUDYhXwkmfOly8iTdp5TEcJFWZD2D7SIkUc=
github.com/montanaflynn/stats v0.5.0 h1:2EkzeTSqBB4V4bJwWrt5gIIrZmpJBcoIRGS2kWLgzmk=
github.com/montanaflynn/stats v0.5.0/go.mod h1:wL8QJuTMNUDYhXwkmfOly8iTdp5TEcJFWZD2D7SIkUc=
github.com/pborman/uuid v0.0.0-20180906182336-adf5a7427709 h1:zNBQb37RGLmJybyMcs983HfUfpkw9OTFD9tbBfAViHE= github.com/pborman/uuid v0.0.0-20180906182336-adf5a7427709 h1:zNBQb37RGLmJybyMcs983HfUfpkw9OTFD9tbBfAViHE=
github.com/pborman/uuid v0.0.0-20180906182336-adf5a7427709/go.mod h1:VyrYX9gd7irzKovcSS6BIIEwPRkP2Wm2m9ufcdFSJ34= github.com/pborman/uuid v0.0.0-20180906182336-adf5a7427709/go.mod h1:VyrYX9gd7irzKovcSS6BIIEwPRkP2Wm2m9ufcdFSJ34=
github.com/pborman/uuid v1.2.0 h1:J7Q5mO4ysT1dv8hyrUGHb9+ooztCXu1D8MY8DZYsu3g=
github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
github.com/pkg/errors v0.8.0 h1:WdK/asTD0HN+q6hsWO3/vpuAkAr+tw6aNJNDFFf0+qw= github.com/pkg/errors v0.8.0 h1:WdK/asTD0HN+q6hsWO3/vpuAkAr+tw6aNJNDFFf0+qw=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/samuel/go-zookeeper v0.0.0-20180130194729-c4fab1ac1bec h1:6ncX5ko6B9LntYM0YBRXkiSaZMmLYeZ/NWcmeB43mMY= github.com/samuel/go-zookeeper v0.0.0-20180130194729-c4fab1ac1bec h1:6ncX5ko6B9LntYM0YBRXkiSaZMmLYeZ/NWcmeB43mMY=
github.com/samuel/go-zookeeper v0.0.0-20180130194729-c4fab1ac1bec/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E= github.com/samuel/go-zookeeper v0.0.0-20180130194729-c4fab1ac1bec/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E=
github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da h1:p3Vo3i64TCLY7gIfzeQaUJ+kppEO5WQG3cL8iE8tGHU=
github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E=
github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk= github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
gitlab.com/spdf/elektron v0.0.0-20191024200717-26f96f361f10 h1:8ySmVAfFtX2mY2o7vbJauJBflwlDk1+XA6Xlp2K6bRg=
gitlab.com/spdf/elektron v0.0.0-20191024200717-26f96f361f10/go.mod h1:F6knSIgL9ajNjb0oNAjzV0vpJ51FtNs1UHxw86QQAQ0=
golang.org/x/crypto v0.0.0-20180927165925-5295e8364332 h1:hvQVdF6P9DX4OiKA5tpehlG6JsgzmyQiThG7q5Bn3UQ= golang.org/x/crypto v0.0.0-20180927165925-5295e8364332 h1:hvQVdF6P9DX4OiKA5tpehlG6JsgzmyQiThG7q5Bn3UQ=
golang.org/x/crypto v0.0.0-20180927165925-5295e8364332/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20180927165925-5295e8364332/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190927123631-a832865fa7ad h1:5E5raQxcv+6CZ11RrBYQe5WRbUIWpScjh0kvHZkZIrQ=
golang.org/x/crypto v0.0.0-20190927123631-a832865fa7ad/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/net v0.0.0-20180926154720-4dfa2610cdf3 h1:dgd4x4kJt7G4k4m93AYLzM8Ni6h2qLTfh9n9vXJT3/0= golang.org/x/net v0.0.0-20180926154720-4dfa2610cdf3 h1:dgd4x4kJt7G4k4m93AYLzM8Ni6h2qLTfh9n9vXJT3/0=
golang.org/x/net v0.0.0-20180926154720-4dfa2610cdf3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180926154720-4dfa2610cdf3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3 h1:0GoQqolDA55aaLxZyTzK/Y2ePZzZTUrRacwib7cNsYQ=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sys v0.0.0-20180928133829-e4b3c5e90611/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180928133829-e4b3c5e90611/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a h1:aYOabOQFp6Vj6W1F80affTUvO9UxmJRx8K0gsfABByQ=
golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw= gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=

26
logConfig.yaml Normal file
View file

@ -0,0 +1,26 @@
schedTrace:
enabled: true
filenameExtension: _schedTrace.log
allowOnConsole: true
sps:
enabled: false
filenameExtension: _schedPolicySwitch.log
allowOnConsole: true
console:
enabled: true
filenameExtension: _console.log
minLogLevel: debug
allowOnConsole: true
pcp:
enabled: true
filenameExtension: .pcplog
allowOnConsole: false
schedWindow:
enabled: false
filenameExtension: _schedWindow.log
allowOnConsole: true
clsfnTaskDistrOverhead:
enabled: false
filenameExtension: _classificationOverhead.log
allowOnConsole: true

117
logging/clsfnTaskDistrOverheadLogger.go Normal file
View file

@ -0,0 +1,117 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"os"
"path/filepath"
"strings"
log "github.com/sirupsen/logrus"
)
type clsfnTaskDistrOverheadLogger struct {
baseElektronLogger
}
func newClsfnTaskDistrOverheadLogger(
config *loggerConfig,
b *baseLogData,
logType int,
prefix string,
logger *log.Logger,
logDir *logDirectory) *clsfnTaskDistrOverheadLogger {
cLog := &clsfnTaskDistrOverheadLogger{
baseElektronLogger: baseElektronLogger{
baseLogData: b,
config: struct {
Enabled bool
FilenameExtension string
AllowOnConsole bool
}{
Enabled: config.TaskDistrConfig.Enabled,
FilenameExtension: config.TaskDistrConfig.FilenameExtension,
AllowOnConsole: config.TaskDistrConfig.AllowOnConsole,
},
logType: logType,
next: nil,
logger: logger,
logDir: logDir,
},
}
cLog.createLogFile(prefix)
return cLog
}
func (cLog clsfnTaskDistrOverheadLogger) Log(logType int, level log.Level, message string) {
if cLog.logType == logType {
if cLog.isEnabled() {
if cLog.isAllowedOnConsole() {
cLog.logger.SetOutput(os.Stdout)
cLog.logger.WithFields(cLog.data).Log(level, message)
}
cLog.logger.SetOutput(cLog.logFile)
cLog.logger.WithFields(cLog.data).Log(level, message)
}
}
// Forwarding to next logger
if cLog.next != nil {
cLog.next.Log(logType, level, message)
} else {
// Clearing the fields.
cLog.resetFields()
}
}
func (cLog clsfnTaskDistrOverheadLogger) Logf(logType int, level log.Level, msgFmtString string, args ...interface{}) {
if cLog.logType == logType {
if cLog.isEnabled() {
if cLog.isAllowedOnConsole() {
cLog.logger.SetOutput(os.Stdout)
cLog.logger.WithFields(cLog.data).Logf(level, msgFmtString, args...)
}
cLog.logger.SetOutput(cLog.logFile)
cLog.logger.WithFields(cLog.data).Logf(level, msgFmtString, args...)
}
}
if cLog.next != nil {
cLog.next.Logf(logType, level, msgFmtString, args...)
} else {
// Clearing the fields.
cLog.resetFields()
}
}
func (cLog *clsfnTaskDistrOverheadLogger) createLogFile(prefix string) {
if cLog.isEnabled() {
filename := strings.Join([]string{prefix, cLog.getFilenameExtension()}, "")
dirName := cLog.logDir.getDirName()
if dirName != "" {
if logFile, err := os.Create(filepath.Join(dirName, filename)); err != nil {
log.Fatal("Unable to create logFile: ", err)
} else {
cLog.logFile = logFile
}
}
}
}

117
logging/consoleLogger.go Normal file
View file

@ -0,0 +1,117 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"os"
"path/filepath"
"strings"
log "github.com/sirupsen/logrus"
)
type consoleLogger struct {
baseElektronLogger
MinLogLevel string
}
func newConsoleLogger(
config *loggerConfig,
b *baseLogData,
logType int,
prefix string,
logger *log.Logger,
logDir *logDirectory) *consoleLogger {
cLog := &consoleLogger{
baseElektronLogger: baseElektronLogger{
baseLogData: b,
config: struct {
Enabled bool
FilenameExtension string
AllowOnConsole bool
}{
Enabled: config.ConsoleConfig.Enabled,
FilenameExtension: config.ConsoleConfig.FilenameExtension,
AllowOnConsole: config.ConsoleConfig.AllowOnConsole,
},
logType: logType,
next: nil,
logger: logger,
logDir: logDir,
},
MinLogLevel: config.ConsoleConfig.MinLogLevel,
}
cLog.createLogFile(prefix)
return cLog
}
func (cLog consoleLogger) Log(logType int, level log.Level, message string) {
if logType <= cLog.logType {
if cLog.isEnabled() {
cLog.logger.SetOutput(os.Stdout)
cLog.logger.WithFields(cLog.data).Log(level, message)
cLog.logger.SetOutput(cLog.logFile)
cLog.logger.WithFields(cLog.data).Log(level, message)
}
}
// Forwarding to next logger.
if cLog.next != nil {
cLog.next.Log(logType, level, message)
} else {
// Clearing the fields.
cLog.resetFields()
}
}
func (cLog consoleLogger) Logf(logType int, level log.Level, msgFmtString string, args ...interface{}) {
if logType <= cLog.logType {
if cLog.isEnabled() {
cLog.logger.SetOutput(os.Stdout)
cLog.logger.WithFields(cLog.data).Logf(level, msgFmtString, args...)
cLog.logger.SetOutput(cLog.logFile)
cLog.logger.WithFields(cLog.data).Logf(level, msgFmtString, args...)
}
}
if cLog.next != nil {
cLog.next.Logf(logType, level, msgFmtString, args...)
} else {
// Clearing the fields.
cLog.resetFields()
}
}
func (cLog *consoleLogger) createLogFile(prefix string) {
// Create log file for the type if it is enabled.
if cLog.isEnabled() {
filename := strings.Join([]string{prefix, cLog.getFilenameExtension()}, "")
dirName := cLog.logDir.getDirName()
if dirName != "" {
if logFile, err := os.Create(filepath.Join(dirName, filename)); err != nil {
log.Fatal("Unable to create logFile: ", err)
} else {
cLog.logFile = logFile
}
}
}
}

57
logging/createLogDir.go Normal file
View file

@ -0,0 +1,57 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"os"
"strconv"
"strings"
"time"
log "github.com/sirupsen/logrus"
)
type logDirectory struct {
name string
}
func (logD *logDirectory) getDirName() string {
return logD.name
}
func (logD *logDirectory) createLogDir(prefix string, startTime time.Time) {
if logD.name == "" {
// Creating directory to store all logs for this run. Directory name format : _2019-November-21_14-33-0.
prefix = prefix + "_"
logDirName := strings.Join([]string{"./", prefix, strconv.Itoa(startTime.Year())}, "")
logDirName = strings.Join([]string{logDirName, startTime.Month().String(), strconv.Itoa(startTime.Day())}, "-")
logDirName = strings.Join([]string{logDirName, strconv.Itoa(startTime.Hour())}, "_")
logDirName = strings.Join([]string{logDirName, strconv.Itoa(startTime.Minute()), strconv.Itoa(startTime.Second())}, "-")
if _, err := os.Stat(logDirName); os.IsNotExist(err) {
os.Mkdir(logDirName, 0755)
} else {
log.Println("Unable to create log directory: ", err)
logDirName = ""
}
logD.name = logDirName
}
}

29
logging/def/clsfnTaskDistOverhead.go
View file

@ -1,29 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
type ClsfnTaskDistOverheadLogger struct {
loggerObserverImpl
}
func (col ClsfnTaskDistOverheadLogger) Log(message string) {
// Logging the overhead of classifying tasks in the scheduling window and determining the distribution
// of light power consuming and heavy power consuming tasks.
col.logObserverSpecifics[clsfnTaskDistOverheadLogger].logFile.Println(message)
}

36
logging/def/consoleLogger.go
View file

@ -1,36 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"log"
)
type ConsoleLogger struct {
loggerObserverImpl
}
func (cl ConsoleLogger) Log(message string) {
// We need to log to console only if the message is not empty
if message != "" {
log.Println(message)
// Also logging the message to the console log file
cl.logObserverSpecifics[conLogger].logFile.Println(message)
}
}

62
logging/def/logType.go
View file

@ -1,62 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import "github.com/fatih/color"
// Defining enums of log message types
var logMessageNames []string
// Possible log message types
var (
ERROR = messageNametoMessageType("ERROR")
WARNING = messageNametoMessageType("WARNING")
GENERAL = messageNametoMessageType("GENERAL")
SUCCESS = messageNametoMessageType("SUCCESS")
SCHED_TRACE = messageNametoMessageType("SCHED_TRACE")
PCP = messageNametoMessageType("PCP")
SPS = messageNametoMessageType("SPS")
CLSFN_TASKDIST_OVERHEAD = messageNametoMessageType("CLSFN_TASKDIST_OVERHEAD")
SCHED_WINDOW = messageNametoMessageType("SCHED_WINDOW")
)
// Text colors for the different types of log messages.
var LogMessageColors map[LogMessageType]*color.Color = map[LogMessageType]*color.Color{
ERROR: color.New(color.FgRed, color.Bold),
WARNING: color.New(color.FgYellow, color.Bold),
GENERAL: color.New(color.FgWhite, color.Bold),
SUCCESS: color.New(color.FgGreen, color.Bold),
}
type LogMessageType int
func (lmt LogMessageType) String() string {
return logMessageNames[lmt]
}
func GetLogMessageTypes() []string {
return logMessageNames
}
func messageNametoMessageType(messageName string) LogMessageType {
// Appending messageName to LogMessageNames
logMessageNames = append(logMessageNames, messageName)
// Mapping messageName to int
return LogMessageType(len(logMessageNames) - 1)
}

77
logging/def/logger.go
View file

@ -1,77 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"time"
)
type LoggerDriver struct {
loggerSubject
allowedMessageTypes map[LogMessageType]bool
}
func newLogger() *LoggerDriver {
logger := &LoggerDriver{
allowedMessageTypes: map[LogMessageType]bool{
ERROR: true,
GENERAL: true,
WARNING: true,
SCHED_TRACE: true,
SUCCESS: true,
PCP: true,
SPS: true,
CLSFN_TASKDIST_OVERHEAD: true,
SCHED_WINDOW: true,
},
}
return logger
}
func BuildLogger(startTime time.Time, prefix string) *LoggerDriver {
// building logger
l := newLogger()
attachAllLoggers(l, startTime, prefix)
return l
}
func (log *LoggerDriver) EnabledLogging(messageType LogMessageType) {
log.allowedMessageTypes[messageType] = true
}
func (log *LoggerDriver) DisableLogging(messageType LogMessageType) {
log.allowedMessageTypes[messageType] = false
}
func (log *LoggerDriver) WriteLog(messageType LogMessageType, message string) {
// checking to see if logging for given messageType is disabled
if log.allowedMessageTypes[messageType] {
log.setMessage(message)
// notify registered loggers to log
log.notify(messageType)
}
}
func (log *LoggerDriver) Listen(logMType <-chan LogMessageType, logMsg <-chan string) {
for {
mType := <-logMType
msg := <-logMsg
log.WriteLog(mType, msg)
}
}

121
logging/def/loggerFactory.go
View file

@ -1,121 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"strings"
"time"
logUtils "github.com/spdfg/elektron/logging/utils"
)
// Names of different loggers
const (
conLogger = "console-logger"
schedTraceLogger = "schedTrace-logger"
pcpLogger = "pcp-logger"
spsLogger = "schedPolicySwitch-logger"
clsfnTaskDistOverheadLogger = "classificationOverhead-logger"
schedWindowLogger = "schedWindow-logger"
)
// Logger class factory
var Loggers map[string]loggerObserver = map[string]loggerObserver{
conLogger: nil,
schedTraceLogger: nil,
pcpLogger: nil,
spsLogger: nil,
clsfnTaskDistOverheadLogger: nil,
schedWindowLogger: nil,
}
// Logger options to help initialize loggers
type loggerOption func(l loggerObserver) error
func withLogDirectory(startTime time.Time, prefix string) loggerOption {
return func(l loggerObserver) error {
l.(*loggerObserverImpl).setLogDirectory(logUtils.GetLogDir(startTime, prefix))
return nil
}
}
// This loggerOption initializes the specifics for each loggerObserver
func withLoggerSpecifics(prefix string) loggerOption {
return func(l loggerObserver) error {
l.(*loggerObserverImpl).logObserverSpecifics = map[string]*specifics{
conLogger: &specifics{},
schedTraceLogger: &specifics{},
pcpLogger: &specifics{},
spsLogger: &specifics{},
clsfnTaskDistOverheadLogger: &specifics{},
schedWindowLogger: &specifics{},
}
l.(*loggerObserverImpl).setLogFilePrefix(prefix)
l.(*loggerObserverImpl).setLogFile()
return nil
}
}
// Build and assign all loggers
func attachAllLoggers(lg *LoggerDriver, startTime time.Time, prefix string) {
loi := &loggerObserverImpl{}
loi.init(withLogDirectory(startTime, strings.Split(prefix, startTime.Format("20060102150405"))[0]),
withLoggerSpecifics(prefix))
Loggers[conLogger] = &ConsoleLogger{
loggerObserverImpl: *loi,
}
Loggers[schedTraceLogger] = &SchedTraceLogger{
loggerObserverImpl: *loi,
}
Loggers[pcpLogger] = &PCPLogger{
loggerObserverImpl: *loi,
}
Loggers[spsLogger] = &SchedPolicySwitchLogger{
loggerObserverImpl: *loi,
}
Loggers[clsfnTaskDistOverheadLogger] = &ClsfnTaskDistOverheadLogger{
loggerObserverImpl: *loi,
}
Loggers[schedWindowLogger] = &SchedWindowLogger{
loggerObserverImpl: *loi,
}
for _, lmt := range GetLogMessageTypes() {
switch lmt {
case SCHED_TRACE.String():
lg.attach(SCHED_TRACE, Loggers[schedTraceLogger])
case GENERAL.String():
lg.attach(GENERAL, Loggers[conLogger])
case WARNING.String():
lg.attach(WARNING, Loggers[conLogger])
case ERROR.String():
lg.attach(ERROR, Loggers[conLogger])
case SUCCESS.String():
lg.attach(SUCCESS, Loggers[conLogger])
case PCP.String():
lg.attach(PCP, Loggers[pcpLogger])
case SPS.String():
lg.attach(SPS, Loggers[spsLogger])
case CLSFN_TASKDIST_OVERHEAD.String():
lg.attach(CLSFN_TASKDIST_OVERHEAD, Loggers[clsfnTaskDistOverheadLogger])
case SCHED_WINDOW.String():
lg.attach(SCHED_WINDOW, Loggers[schedWindowLogger])
}
}
}

120
logging/def/loggerObservers.go
View file

@ -1,120 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"fmt"
"log"
"os"
)
// Logging platform
type loggerObserver interface {
Log(message string)
setLogFile()
setLogFilePrefix(prefix string)
setLogDirectory(dirName string)
init(opts ...loggerOption)
}
type specifics struct {
logFilePrefix string
logFile *log.Logger
}
type loggerObserverImpl struct {
logFile *log.Logger
logObserverSpecifics map[string]*specifics
logDirectory string
}
func (loi *loggerObserverImpl) init(opts ...loggerOption) {
for _, opt := range opts {
// applying logger options
if err := opt(loi); err != nil {
log.Fatal(err)
}
}
}
func (loi loggerObserverImpl) Log(message string) {}
// Requires logFilePrefix to have already been set
func (loi *loggerObserverImpl) setLogFile() {
for prefix, ls := range loi.logObserverSpecifics {
if logFile, err := os.Create(ls.logFilePrefix); err != nil {
log.Fatal("Unable to create logFile: ", err)
} else {
fmt.Printf("Creating logFile with pathname: %s, and prefix: %s\n", ls.logFilePrefix, prefix)
ls.logFile = log.New(logFile, "", log.LstdFlags)
}
}
}
func (loi *loggerObserverImpl) setLogFilePrefix(prefix string) {
// Setting logFilePrefix for pcp logger
pcpLogFilePrefix := prefix + ".pcplog"
if loi.logDirectory != "" {
pcpLogFilePrefix = loi.logDirectory + "/" + pcpLogFilePrefix
}
loi.logObserverSpecifics[pcpLogger].logFilePrefix = pcpLogFilePrefix
// Setting logFilePrefix for console logger
consoleLogFilePrefix := prefix + "_console.log"
if loi.logDirectory != "" {
consoleLogFilePrefix = loi.logDirectory + "/" + consoleLogFilePrefix
}
loi.logObserverSpecifics[conLogger].logFilePrefix = consoleLogFilePrefix
// Setting logFilePrefix for schedTrace logger
schedTraceLogFilePrefix := prefix + "_schedTrace.log"
if loi.logDirectory != "" {
schedTraceLogFilePrefix = loi.logDirectory + "/" + schedTraceLogFilePrefix
}
loi.logObserverSpecifics[schedTraceLogger].logFilePrefix = schedTraceLogFilePrefix
// Setting logFilePrefix for schedulingPolicySwitch logger
schedPolicySwitchLogFilePrefix := prefix + "_schedPolicySwitch.log"
if loi.logDirectory != "" {
schedPolicySwitchLogFilePrefix = loi.logDirectory + "/" + schedPolicySwitchLogFilePrefix
}
loi.logObserverSpecifics[spsLogger].logFilePrefix = schedPolicySwitchLogFilePrefix
// Setting logFilePrefix for clsfnTaskDist logger.
// Execution time of every call to def.GetTaskDistribution(...) would be recorded and logged in this file.
// The overhead would be logged in microseconds.
clsfnTaskDistOverheadLogFilePrefix := prefix + "_classificationOverhead.log"
if loi.logDirectory != "" {
clsfnTaskDistOverheadLogFilePrefix = loi.logDirectory + "/" + clsfnTaskDistOverheadLogFilePrefix
}
loi.logObserverSpecifics[clsfnTaskDistOverheadLogger].logFilePrefix = clsfnTaskDistOverheadLogFilePrefix
// Setting logFilePrefix for schedWindow logger.
// Going to log the time stamp when the scheduling window was determined
// and the size of the scheduling window.
schedWindowLogFilePrefix := prefix + "_schedWindow.log"
if loi.logDirectory != "" {
schedWindowLogFilePrefix = loi.logDirectory + "/" + schedWindowLogFilePrefix
}
loi.logObserverSpecifics[schedWindowLogger].logFilePrefix = schedWindowLogFilePrefix
}
func (loi *loggerObserverImpl) setLogDirectory(dirName string) {
loi.logDirectory = dirName
}

41
logging/def/loggerSubject.go
View file

@ -1,41 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
type loggerSubject struct {
Registry map[LogMessageType][]loggerObserver
message string
}
func (ls *loggerSubject) setMessage(message string) {
ls.message = message
}
func (ls *loggerSubject) attach(messageType LogMessageType, lo loggerObserver) {
if ls.Registry == nil {
ls.Registry = make(map[LogMessageType][]loggerObserver)
}
ls.Registry[messageType] = append(ls.Registry[messageType], lo)
}
func (ls *loggerSubject) notify(messageType LogMessageType) {
for _, logObserver := range ls.Registry[messageType] {
logObserver.Log(ls.message)
}
}

27
logging/def/schedPolicySwitchLogger.go
View file

@ -1,27 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
type SchedPolicySwitchLogger struct {
loggerObserverImpl
}
func (pl *SchedPolicySwitchLogger) Log(message string) {
pl.logObserverSpecifics[spsLogger].logFile.Println(message)
}

28
logging/def/schedTraceLogger.go
View file

@ -1,28 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
type SchedTraceLogger struct {
loggerObserverImpl
}
func (stl SchedTraceLogger) Log(message string) {
// Logging schedule trace to mentioned file
stl.logObserverSpecifics[schedTraceLogger].logFile.Println(message)
}

28
logging/def/schedWindowLogger.go
View file

@ -1,28 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
type SchedWindowLogger struct {
loggerObserverImpl
}
func (swl SchedWindowLogger) Log(message string) {
// Logging schedule trace to mentioned file
swl.logObserverSpecifics[schedWindowLogger].logFile.Println(message)
}

54
logging/elektronFormatter.go Normal file
View file

@ -0,0 +1,54 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"bytes"
"fmt"
"strings"
log "github.com/sirupsen/logrus"
)
type elektronFormatter struct {
TimestampFormat string
}
func (f elektronFormatter) Format(entry *log.Entry) ([]byte, error) {
var b *bytes.Buffer
if entry.Buffer != nil {
b = entry.Buffer
} else {
b = &bytes.Buffer{}
}
level := fmt.Sprintf("[%s]:", strings.ToUpper(entry.Level.String()))
message := strings.Join([]string{level, entry.Time.Format(f.TimestampFormat), entry.Message, " "}, " ")
var formattedFields []string
for key, value := range entry.Data {
formattedFields = append(formattedFields, strings.Join([]string{key, value.(string)}, "="))
}
b.WriteString(message)
b.WriteString(strings.Join(formattedFields, ", "))
b.WriteByte('\n')
return b.Bytes(), nil
}

165
logging/logger.go Normal file
View file

@ -0,0 +1,165 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"github.com/pkg/errors"
"os"
"strings"
"time"
log "github.com/sirupsen/logrus"
. "github.com/spdfg/elektron/logging/types"
)
// var config loggerConfig
var formatter elektronFormatter
var elektronLoggerInstance elektronLogger
type elektronLogger interface {
setNext(next elektronLogger)
Log(logType int, level log.Level, message string)
Logf(logType int, level log.Level, msgFmtString string, args ...interface{})
WithFields(logData log.Fields) elektronLogger
WithField(key string, value string) elektronLogger
}
type baseLogData struct {
data log.Fields
}
type baseElektronLogger struct {
*baseLogData
config struct {
Enabled bool
FilenameExtension string
AllowOnConsole bool
}
logType int
logFile *os.File
next elektronLogger
logger *log.Logger
logDir *logDirectory
}
func (l baseElektronLogger) isEnabled() bool {
return l.config.Enabled
}
func (l baseElektronLogger) isAllowedOnConsole() bool {
return l.config.AllowOnConsole
}
func (l baseElektronLogger) getFilenameExtension() string {
return l.config.FilenameExtension
}
func (l *baseElektronLogger) WithFields(logData log.Fields) elektronLogger {
l.data = logData
return l
}
func (l *baseElektronLogger) WithField(key string, value string) elektronLogger {
l.data[key] = value
return l
}
func (l *baseElektronLogger) setNext(next elektronLogger) {
l.next = next
}
func (l baseElektronLogger) Log(logType int, level log.Level, message string) {
if l.next != nil {
l.next.Log(logType, level, message)
}
}
func (l baseElektronLogger) Logf(logType int, level log.Level, msgFmtString string, args ...interface{}) {
if l.next != nil {
l.next.Logf(logType, level, msgFmtString, args...)
}
}
func (l *baseElektronLogger) resetFields() {
l.data = nil
l.data = log.Fields{}
}
func BuildLogger(prefix string, logConfigFilename string) error {
// Create the log directory.
startTime := time.Now()
formatter.TimestampFormat = "2006-01-02 15:04:05"
formattedStartTime := startTime.Format("20060102150405")
logDir := &logDirectory{}
logDir.createLogDir(prefix, startTime)
// Instantiate the logrus instance.
prefix = strings.Join([]string{prefix, formattedStartTime}, "_")
logger := &log.Logger{
Out: os.Stderr,
Level: log.DebugLevel,
Formatter: &formatter,
}
// Create a chain of loggers.
b := &baseLogData{data: log.Fields{}}
head := &baseElektronLogger{baseLogData: b}
// Read configuration from yaml.
if config, err := getConfig(logConfigFilename); err != nil {
return errors.Wrap(err, "Failed to build logger")
} else {
cLog := newConsoleLogger(config, b, CONSOLE, prefix, logger, logDir)
pLog := newPCPLogger(config, b, PCP, prefix, logger, logDir)
schedTraceLog := newSchedTraceLogger(config, b, SCHED_TRACE, prefix, logger, logDir)
spsLog := newSchedPolicySwitchLogger(config, b, SPS, prefix, logger, logDir)
schedWindowLog := newSchedWindowLogger(config, b, SCHED_WINDOW, prefix, logger, logDir)
tskDistLog := newClsfnTaskDistrOverheadLogger(config, b, CLSFN_TASKDISTR_OVERHEAD, prefix, logger, logDir)
head.setNext(cLog)
cLog.setNext(pLog)
pLog.setNext(schedTraceLog)
schedTraceLog.setNext(spsLog)
spsLog.setNext(schedWindowLog)
schedWindowLog.setNext(tskDistLog)
}
elektronLoggerInstance = head
return nil
}
func Log(logType int, level log.Level, message string) {
elektronLoggerInstance.Log(logType, level, message)
}
func Logf(logType int, level log.Level, msgFmtString string, args ...interface{}) {
elektronLoggerInstance.Logf(logType, level, msgFmtString, args...)
}
func WithFields(logData log.Fields) elektronLogger {
return elektronLoggerInstance.WithFields(logData)
}
func WithField(key string, value string) elektronLogger {
return elektronLoggerInstance.WithField(key, value)
}

83
logging/loggerConfig.go Normal file
View file

@ -0,0 +1,83 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
"gopkg.in/yaml.v2"
"io/ioutil"
)
type loggerConfig struct {
SchedTraceConfig struct {
Enabled bool `yaml:"enabled"`
FilenameExtension string `yaml:"filenameExtension"`
AllowOnConsole bool `yaml:"allowOnConsole"`
} `yaml:"schedTrace"`
PCPConfig struct {
Enabled bool `yaml:"enabled"`
FilenameExtension string `yaml:"filenameExtension"`
AllowOnConsole bool `yaml:"allowOnConsole"`
} `yaml:"pcp"`
ConsoleConfig struct {
Enabled bool `yaml:"enabled"`
FilenameExtension string `yaml:"filenameExtension"`
MinLogLevel string `yaml:"minLogLevel"`
AllowOnConsole bool `yaml:"allowOnConsole"`
} `yaml:"console"`
SPSConfig struct {
Enabled bool `yaml:"enabled"`
FilenameExtension string `yaml:"filenameExtension"`
AllowOnConsole bool `yaml:"allowOnConsole"`
} `yaml:"sps"`
TaskDistrConfig struct {
Enabled bool `yaml:"enabled"`
FilenameExtension string `yaml:"filenameExtension"`
AllowOnConsole bool `yaml:"allowOnConsole"`
} `yaml:"clsfnTaskDistrOverhead"`
SchedWindowConfig struct {
Enabled bool `yaml:"enabled"`
FilenameExtension string `yaml:"filenameExtension"`
AllowOnConsole bool `yaml:"allowOnConsole"`
} `yaml:"schedWindow"`
Format []string `yaml:"format"`
}
func getConfig(logConfigFilename string) (*loggerConfig, error) {
yamlFile, err := ioutil.ReadFile(logConfigFilename)
if err != nil {
return nil, errors.Wrap(err, "failed to read log config file")
}
c := &loggerConfig{}
err = yaml.Unmarshal(yamlFile, c)
if err != nil {
log.Fatalf("Error in unmarshalling yaml: %v", err)
}
return c, nil
}

117
logging/pcpLogger.go Normal file
View file

@ -0,0 +1,117 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"os"
"path/filepath"
"strings"
log "github.com/sirupsen/logrus"
)
type pcpLogger struct {
baseElektronLogger
}
func newPCPLogger(
config *loggerConfig,
b *baseLogData,
logType int,
prefix string,
logger *log.Logger,
logDir *logDirectory) *pcpLogger {
pLog := &pcpLogger{
baseElektronLogger: baseElektronLogger{
baseLogData: b,
config: struct {
Enabled bool
FilenameExtension string
AllowOnConsole bool
}{
Enabled: config.PCPConfig.Enabled,
FilenameExtension: config.PCPConfig.FilenameExtension,
AllowOnConsole: config.PCPConfig.AllowOnConsole,
},
logType: logType,
next: nil,
logger: logger,
logDir: logDir,
},
}
pLog.createLogFile(prefix)
return pLog
}
func (pLog pcpLogger) Log(logType int, level log.Level, message string) {
if pLog.logType == logType {
if pLog.isEnabled() {
if pLog.isAllowedOnConsole() {
pLog.logger.SetOutput(os.Stdout)
pLog.logger.WithFields(pLog.data).Log(level, message)
}
pLog.logger.SetOutput(pLog.logFile)
pLog.logger.WithFields(pLog.data).Log(level, message)
}
}
if pLog.next != nil {
pLog.next.Log(logType, level, message)
} else {
// Clearing the fields.
pLog.resetFields()
}
}
func (pLog pcpLogger) Logf(logType int, level log.Level, msgFmtString string, args ...interface{}) {
if pLog.logType == logType {
if pLog.isEnabled() {
if pLog.isAllowedOnConsole() {
pLog.logger.SetOutput(os.Stdout)
pLog.logger.WithFields(pLog.data).Logf(level, msgFmtString, args...)
}
pLog.logger.SetOutput(pLog.logFile)
pLog.logger.WithFields(pLog.data).Logf(level, msgFmtString, args...)
}
}
// Forwarding to next logger
if pLog.next != nil {
pLog.next.Logf(logType, level, msgFmtString, args...)
} else {
// Clearing the fields.
pLog.resetFields()
}
}
func (pLog *pcpLogger) createLogFile(prefix string) {
if pLog.isEnabled() {
filename := strings.Join([]string{prefix, pLog.getFilenameExtension()}, "")
dirName := pLog.logDir.getDirName()
if dirName != "" {
if logFile, err := os.Create(filepath.Join(dirName, filename)); err != nil {
log.Fatal("Unable to create logFile: ", err)
} else {
pLog.logFile = logFile
}
}
}
}

117
logging/schedPolicySwitchLogger.go Normal file
View file

@ -0,0 +1,117 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"os"
"path/filepath"
"strings"
log "github.com/sirupsen/logrus"
)
type schedPolicySwitchLogger struct {
baseElektronLogger
}
func newSchedPolicySwitchLogger(
config *loggerConfig,
b *baseLogData,
logType int,
prefix string,
logger *log.Logger,
logDir *logDirectory) *schedPolicySwitchLogger {
sLog := &schedPolicySwitchLogger{
baseElektronLogger: baseElektronLogger{
baseLogData: b,
config: struct {
Enabled bool
FilenameExtension string
AllowOnConsole bool
}{
Enabled: config.SPSConfig.Enabled,
FilenameExtension: config.SPSConfig.FilenameExtension,
AllowOnConsole: config.SPSConfig.AllowOnConsole,
},
logType: logType,
next: nil,
logger: logger,
logDir: logDir,
},
}
sLog.createLogFile(prefix)
return sLog
}
func (sLog schedPolicySwitchLogger) Log(logType int, level log.Level, message string) {
if sLog.logType == logType {
if sLog.isEnabled() {
if sLog.isAllowedOnConsole() {
sLog.logger.SetOutput(os.Stdout)
sLog.logger.WithFields(sLog.data).Log(level, message)
}
sLog.logger.SetOutput(sLog.logFile)
sLog.logger.WithFields(sLog.data).Log(level, message)
}
}
if sLog.next != nil {
sLog.next.Log(logType, level, message)
} else {
// Clearing the fields.
sLog.resetFields()
}
}
func (sLog schedPolicySwitchLogger) Logf(logType int, level log.Level, msgFmtString string, args ...interface{}) {
if sLog.logType == logType {
if sLog.isEnabled() {
if sLog.isAllowedOnConsole() {
sLog.logger.SetOutput(os.Stdout)
sLog.logger.WithFields(sLog.data).Logf(level, msgFmtString, args...)
}
sLog.logger.SetOutput(sLog.logFile)
sLog.logger.WithFields(sLog.data).Logf(level, msgFmtString, args...)
}
}
// Forwarding to next logger
if sLog.next != nil {
sLog.next.Logf(logType, level, msgFmtString, args...)
} else {
// Clearing the fields.
sLog.resetFields()
}
}
func (sLog *schedPolicySwitchLogger) createLogFile(prefix string) {
if sLog.isEnabled() {
filename := strings.Join([]string{prefix, sLog.getFilenameExtension()}, "")
dirName := sLog.logDir.getDirName()
if dirName != "" {
if logFile, err := os.Create(filepath.Join(dirName, filename)); err != nil {
log.Fatal("Unable to create logFile: ", err)
} else {
sLog.logFile = logFile
}
}
}
}

117
logging/schedTraceLogger.go Normal file
View file

@ -0,0 +1,117 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"os"
"path/filepath"
"strings"
log "github.com/sirupsen/logrus"
)
type schedTraceLogger struct {
baseElektronLogger
}
func newSchedTraceLogger(
config *loggerConfig,
b *baseLogData,
logType int,
prefix string,
logger *log.Logger,
logDir *logDirectory) *schedTraceLogger {
sLog := &schedTraceLogger{
baseElektronLogger: baseElektronLogger{
baseLogData: b,
config: struct {
Enabled bool
FilenameExtension string
AllowOnConsole bool
}{
Enabled: config.SchedTraceConfig.Enabled,
FilenameExtension: config.SchedTraceConfig.FilenameExtension,
AllowOnConsole: config.SchedTraceConfig.AllowOnConsole,
},
logType: logType,
next: nil,
logger: logger,
logDir: logDir,
},
}
sLog.createLogFile(prefix)
return sLog
}
func (sLog schedTraceLogger) Log(logType int, level log.Level, message string) {
if sLog.logType == logType {
if sLog.isEnabled() {
if sLog.isAllowedOnConsole() {
sLog.logger.SetOutput(os.Stdout)
sLog.logger.WithFields(sLog.data).Log(level, message)
}
sLog.logger.SetOutput(sLog.logFile)
sLog.logger.WithFields(sLog.data).Log(level, message)
}
}
if sLog.next != nil {
sLog.next.Log(logType, level, message)
} else {
// Clearing the fields.
sLog.resetFields()
}
}
func (sLog schedTraceLogger) Logf(logType int, level log.Level, msgFmtString string, args ...interface{}) {
if sLog.logType == logType {
if sLog.isEnabled() {
if sLog.isAllowedOnConsole() {
sLog.logger.SetOutput(os.Stdout)
sLog.logger.WithFields(sLog.data).Logf(level, msgFmtString, args...)
}
sLog.logger.SetOutput(sLog.logFile)
sLog.logger.WithFields(sLog.data).Logf(level, msgFmtString, args...)
}
}
// Forwarding to next logger
if sLog.next != nil {
sLog.next.Logf(logType, level, msgFmtString, args...)
} else {
// Clearing the fields.
sLog.resetFields()
}
}
func (sLog *schedTraceLogger) createLogFile(prefix string) {
if sLog.isEnabled() {
filename := strings.Join([]string{prefix, sLog.getFilenameExtension()}, "")
dirName := sLog.logDir.getDirName()
if dirName != "" {
if logFile, err := os.Create(filepath.Join(dirName, filename)); err != nil {
log.Fatal("Unable to create logFile: ", err)
} else {
sLog.logFile = logFile
}
}
}
}

117
logging/schedWindowLogger.go Normal file
View file

@ -0,0 +1,117 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"os"
"path/filepath"
"strings"
log "github.com/sirupsen/logrus"
)
type schedWindowLogger struct {
baseElektronLogger
}
func newSchedWindowLogger(
config *loggerConfig,
b *baseLogData,
logType int,
prefix string,
logger *log.Logger,
logDir *logDirectory) *schedWindowLogger {
sLog := &schedWindowLogger{
baseElektronLogger: baseElektronLogger{
baseLogData: b,
config: struct {
Enabled bool
FilenameExtension string
AllowOnConsole bool
}{
Enabled: config.SchedWindowConfig.Enabled,
FilenameExtension: config.SchedWindowConfig.FilenameExtension,
AllowOnConsole: config.SchedWindowConfig.AllowOnConsole,
},
logType: logType,
next: nil,
logger: logger,
logDir: logDir,
},
}
sLog.createLogFile(prefix)
return sLog
}
func (sLog schedWindowLogger) Log(logType int, level log.Level, message string) {
if sLog.logType == logType {
if sLog.isEnabled() {
if sLog.isAllowedOnConsole() {
sLog.logger.SetOutput(os.Stdout)
sLog.logger.WithFields(sLog.data).Log(level, message)
}
sLog.logger.SetOutput(sLog.logFile)
sLog.logger.WithFields(sLog.data).Log(level, message)
}
}
// Forwarding to next logger
if sLog.next != nil {
sLog.next.Log(logType, level, message)
} else {
// Clearing the fields.
sLog.resetFields()
}
}
func (sLog schedWindowLogger) Logf(logType int, level log.Level, msgFmtString string, args ...interface{}) {
if sLog.logType == logType {
if sLog.isEnabled() {
if sLog.isAllowedOnConsole() {
sLog.logger.SetOutput(os.Stdout)
sLog.logger.WithFields(sLog.data).Logf(level, msgFmtString, args...)
}
sLog.logger.SetOutput(sLog.logFile)
sLog.logger.WithFields(sLog.data).Logf(level, msgFmtString, args...)
}
}
if sLog.next != nil {
sLog.next.Logf(logType, level, msgFmtString, args...)
} else {
// Clearing the fields.
sLog.resetFields()
}
}
func (sLog *schedWindowLogger) createLogFile(prefix string) {
if sLog.isEnabled() {
filename := strings.Join([]string{prefix, sLog.getFilenameExtension()}, "")
dirName := sLog.logDir.getDirName()
if dirName != "" {
if logFile, err := os.Create(filepath.Join(dirName, filename)); err != nil {
log.Fatal("Unable to create logFile: ", err)
} else {
sLog.logFile = logFile
}
}
}
}

27
logging/def/pcpLogger.go → logging/types/types.go
View file

@ -1,27 +1,28 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package logging package types
type PCPLogger struct { const (
loggerObserverImpl CONSOLE = iota
} PCP
SCHED_TRACE
func (pl *PCPLogger) Log(message string) { SPS
pl.logObserverSpecifics[pcpLogger].logFile.Println(message) SCHED_WINDOW
} CLSFN_TASKDISTR_OVERHEAD
)

57
logging/utils/createLogDir.go
View file

@ -1,57 +0,0 @@
// Copyright (C) 2018 spdfg
//
// This file is part of Elektron.
//
// Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>.
//
package logging
import (
"log"
"os"
"strconv"
"time"
)
var LogDir string
func GetLogDir(startTime time.Time, prefix string) string {
if LogDir == "" {
LogDir = createLogDir(prefix, startTime)
}
return LogDir
}
func createLogDir(prefix string, startTime time.Time) string {
// Creating directory to store all logs for this run
logDirName := "./" + prefix + strconv.Itoa(startTime.Year())
logDirName += "-"
logDirName += startTime.Month().String()
logDirName += "-"
logDirName += strconv.Itoa(startTime.Day())
logDirName += "_"
logDirName += strconv.Itoa(startTime.Hour())
logDirName += "-"
logDirName += strconv.Itoa(startTime.Minute())
logDirName += "-"
logDirName += strconv.Itoa(startTime.Second())
if _, err := os.Stat(logDirName); os.IsNotExist(err) {
os.Mkdir(logDirName, 0755)
} else {
log.Println("Unable to create log directory: ", err)
logDirName = ""
}
return logDirName
}

30
pcp/pcp.go
View file

@ -1,35 +1,35 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package pcp package pcp
import ( import (
"bufio" "bufio"
"log"
"os/exec" "os/exec"
"syscall" "syscall"
"time" "time"
elekLogDef "github.com/spdfg/elektron/logging/def" log "github.com/sirupsen/logrus"
elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
) )
func Start(quit chan struct{}, logging *bool, logMType chan elekLogDef.LogMessageType, func Start(quit chan struct{}, logging *bool, pcpConfigFile string) {
logMsg chan string, pcpConfigFile string) {
var pcpCommand string = "pmdumptext -m -l -f '' -t 1.0 -d , -c " + pcpConfigFile var pcpCommand string = "pmdumptext -m -l -f '' -t 1.0 -d , -c " + pcpConfigFile
cmd := exec.Command("sh", "-c", pcpCommand) cmd := exec.Command("sh", "-c", pcpCommand)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
@ -47,8 +47,7 @@ func Start(quit chan struct{}, logging *bool, logMType chan elekLogDef.LogMessag
scanner.Scan() scanner.Scan()
// Write to logfile // Write to logfile
logMType <- elekLogDef.PCP elekLog.Log(PCP, log.InfoLevel, scanner.Text())
logMsg <- scanner.Text()
// Throw away first set of results // Throw away first set of results
scanner.Scan() scanner.Scan()
@ -59,16 +58,14 @@ func Start(quit chan struct{}, logging *bool, logMType chan elekLogDef.LogMessag
text := scanner.Text() text := scanner.Text()
if *logging { if *logging {
logMType <- elekLogDef.PCP elekLog.Log(PCP, log.InfoLevel, text)
logMsg <- text
} }
seconds++ seconds++
} }
}(logging) }(logging)
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "PCP logging started")
logMsg <- "PCP logging started"
if err := cmd.Start(); err != nil { if err := cmd.Start(); err != nil {
log.Fatal(err) log.Fatal(err)
@ -78,8 +75,7 @@ func Start(quit chan struct{}, logging *bool, logMType chan elekLogDef.LogMessag
select { select {
case <-quit: case <-quit:
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "Stopping PCP logging in 5 seconds")
logMsg <- "Stopping PCP logging in 5 seconds"
time.Sleep(5 * time.Second) time.Sleep(5 * time.Second)
// http://stackoverflow.com/questions/22470193/why-wont-go-kill-a-child-process-correctly // http://stackoverflow.com/questions/22470193/why-wont-go-kill-a-child-process-correctly

10
pcp/utils.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package pcp package pcp

10
pcp/victim.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package pcp package pcp

67
powerCap/extrema.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package powerCap package powerCap
@ -22,7 +22,6 @@ import (
"bufio" "bufio"
"container/ring" "container/ring"
"fmt" "fmt"
"log"
"os/exec" "os/exec"
"sort" "sort"
"strconv" "strconv"
@ -30,20 +29,21 @@ import (
"syscall" "syscall"
"time" "time"
elekLogDef "github.com/spdfg/elektron/logging/def" log "github.com/sirupsen/logrus"
elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
"github.com/spdfg/elektron/pcp" "github.com/spdfg/elektron/pcp"
"github.com/spdfg/elektron/rapl" "github.com/spdfg/elektron/rapl"
) )
func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThreshold, loThreshold float64, func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThreshold, loThreshold float64, pcpConfigFile string) {
logMType chan elekLogDef.LogMessageType, logMsg chan string, pcpConfigFile string) {
var pcpCommand string = "pmdumptext -m -l -f '' -t 1.0 -d , -c " + pcpConfigFile var pcpCommand string = "pmdumptext -m -l -f '' -t 1.0 -d , -c " + pcpConfigFile
cmd := exec.Command("sh", "-c", pcpCommand, pcpConfigFile) cmd := exec.Command("sh", "-c", pcpCommand, pcpConfigFile)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
if hiThreshold < loThreshold { if hiThreshold < loThreshold {
log.Println("High threshold is lower than low threshold!") elekLog.Log(CONSOLE, log.InfoLevel, "High threshold is lower than low threshold!")
} }
pipe, err := cmd.StdoutPipe() pipe, err := cmd.StdoutPipe()
@ -59,8 +59,7 @@ func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThresh
scanner.Scan() scanner.Scan()
// Write to logfile // Write to logfile
logMType <- elekLogDef.PCP elekLog.Log(PCP, log.InfoLevel, scanner.Text())
logMsg <- scanner.Text()
headers := strings.Split(scanner.Text(), ",") headers := strings.Split(scanner.Text(), ",")
@ -95,12 +94,13 @@ func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThresh
for scanner.Scan() { for scanner.Scan() {
if *logging { if *logging {
logMType <- elekLogDef.GENERAL
logMsg <- "Logging PCP..." elekLog.Log(CONSOLE, log.InfoLevel, "Logging PCP...")
text := scanner.Text() text := scanner.Text()
split := strings.Split(text, ",") split := strings.Split(text, ",")
logMType <- elekLogDef.PCP
logMsg <- text elekLog.Log(PCP, log.InfoLevel, text)
totalPower := 0.0 totalPower := 0.0
for _, powerIndex := range powerIndexes { for _, powerIndex := range powerIndexes {
@ -111,8 +111,10 @@ func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThresh
powerHistories[host].Value = power powerHistories[host].Value = power
powerHistories[host] = powerHistories[host].Next() powerHistories[host] = powerHistories[host].Next()
logMType <- elekLogDef.GENERAL elekLog.WithFields(log.Fields{
logMsg <- fmt.Sprintf("Host: %s, Power: %f", indexToHost[powerIndex], (power * pcp.RAPLUnits)) "Host": indexToHost[powerIndex],
"Power": fmt.Sprintf("%f", power*pcp.RAPLUnits),
}).Log(CONSOLE, log.InfoLevel, "")
totalPower += power totalPower += power
} }
@ -123,12 +125,13 @@ func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThresh
clusterMean := pcp.AverageClusterPowerHistory(clusterPowerHist) clusterMean := pcp.AverageClusterPowerHistory(clusterPowerHist)
logMType <- elekLogDef.GENERAL elekLog.WithFields(log.Fields{
logMsg <- fmt.Sprintf("Total power: %f, %d Sec Avg: %f", clusterPower, clusterPowerHist.Len(), clusterMean) "Total power": fmt.Sprintf("%f %d Sec", clusterPower, clusterPowerHist.Len()),
"Avg": fmt.Sprintf("%f", clusterMean),
}).Log(CONSOLE, log.InfoLevel, "")
if clusterMean > hiThreshold { if clusterMean > hiThreshold {
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "Need to cap a node")
logMsg <- "Need to cap a node"
// Create statics for all victims and choose one to cap // Create statics for all victims and choose one to cap
victims := make([]pcp.Victim, 0, 8) victims := make([]pcp.Victim, 0, 8)
@ -149,11 +152,10 @@ func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThresh
if !cappedHosts[victim.Host] { if !cappedHosts[victim.Host] {
cappedHosts[victim.Host] = true cappedHosts[victim.Host] = true
orderCapped = append(orderCapped, victim.Host) orderCapped = append(orderCapped, victim.Host)
logMType <- elekLogDef.GENERAL elekLog.WithField("Avg. Wattage",
logMsg <- fmt.Sprintf("Capping Victim %s Avg. Wattage: %f", victim.Host, victim.Watts*pcp.RAPLUnits) fmt.Sprintf("%f", victim.Watts*pcp.RAPLUnits)).Logf(CONSOLE, log.InfoLevel, "Capping Victim %s", victim.Host)
if err := rapl.Cap(victim.Host, "rapl", 50); err != nil { if err := rapl.Cap(victim.Host, "rapl", 50); err != nil {
logMType <- elekLogDef.ERROR elekLog.Log(CONSOLE, log.ErrorLevel, "Error capping host")
logMsg <- "Error capping host"
} }
break // Only cap one machine at at time. break // Only cap one machine at at time.
} }
@ -166,12 +168,9 @@ func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThresh
orderCapped = orderCapped[:len(orderCapped)-1] orderCapped = orderCapped[:len(orderCapped)-1]
cappedHosts[host] = false cappedHosts[host] = false
// User RAPL package to send uncap. // User RAPL package to send uncap.
log.Printf("Uncapping host %s", host) elekLog.Logf(CONSOLE, log.InfoLevel, "Uncapping host %s", host)
logMType <- elekLogDef.GENERAL
logMsg <- fmt.Sprintf("Uncapped host %s", host)
if err := rapl.Cap(host, "rapl", 100); err != nil { if err := rapl.Cap(host, "rapl", 100); err != nil {
logMType <- elekLogDef.ERROR elekLog.Log(CONSOLE, log.ErrorLevel, "Error capping host")
logMsg <- "Error capping host"
} }
} }
} }
@ -181,8 +180,7 @@ func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThresh
} }
}(logging, hiThreshold, loThreshold) }(logging, hiThreshold, loThreshold)
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "PCP logging started")
logMsg <- "PCP logging started"
if err := cmd.Start(); err != nil { if err := cmd.Start(); err != nil {
log.Fatal(err) log.Fatal(err)
@ -192,8 +190,7 @@ func StartPCPLogAndExtremaDynamicCap(quit chan struct{}, logging *bool, hiThresh
select { select {
case <-quit: case <-quit:
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "Stopping PCP logging in 5 seconds")
logMsg <- "Stopping PCP logging in 5 seconds"
time.Sleep(5 * time.Second) time.Sleep(5 * time.Second)
// http://stackoverflow.com/questions/22470193/why-wont-go-kill-a-child-process-correctly // http://stackoverflow.com/questions/22470193/why-wont-go-kill-a-child-process-correctly

99
powerCap/progressiveExtrema.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package powerCap package powerCap
@ -22,7 +22,6 @@ import (
"bufio" "bufio"
"container/ring" "container/ring"
"fmt" "fmt"
"log"
"math" "math"
"os/exec" "os/exec"
"sort" "sort"
@ -31,8 +30,10 @@ import (
"syscall" "syscall"
"time" "time"
log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/constants" "github.com/spdfg/elektron/constants"
elekLogDef "github.com/spdfg/elektron/logging/def" elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
"github.com/spdfg/elektron/pcp" "github.com/spdfg/elektron/pcp"
"github.com/spdfg/elektron/rapl" "github.com/spdfg/elektron/rapl"
"github.com/spdfg/elektron/utilities" "github.com/spdfg/elektron/utilities"
@ -48,16 +49,14 @@ func getNextCapValue(curCapValue float64, precision int) float64 {
return float64(round(curCapValue*output)) / output return float64(round(curCapValue*output)) / output
} }
func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiThreshold, loThreshold float64, func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiThreshold, loThreshold float64, pcpConfigFile string) {
logMType chan elekLogDef.LogMessageType, logMsg chan string, pcpConfigFile string) {
var pcpCommand string = "pmdumptext -m -l -f '' -t 1.0 -d , -c " + pcpConfigFile var pcpCommand string = "pmdumptext -m -l -f '' -t 1.0 -d , -c " + pcpConfigFile
cmd := exec.Command("sh", "-c", pcpCommand, pcpConfigFile) cmd := exec.Command("sh", "-c", pcpCommand, pcpConfigFile)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
if hiThreshold < loThreshold { if hiThreshold < loThreshold {
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "High threshold is lower than low threshold!")
logMsg <- "High threshold is lower than low threshold!"
} }
pipe, err := cmd.StdoutPipe() pipe, err := cmd.StdoutPipe()
@ -73,8 +72,7 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
scanner.Scan() scanner.Scan()
// Write to logfile // Write to logfile
logMType <- elekLogDef.PCP elekLog.Log(PCP, log.InfoLevel, scanner.Text())
logMsg <- scanner.Text()
headers := strings.Split(scanner.Text(), ",") headers := strings.Split(scanner.Text(), ",")
@ -113,13 +111,11 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
for scanner.Scan() { for scanner.Scan() {
if *logging { if *logging {
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "Logging PCP...")
logMsg <- "Logging PCP..."
split := strings.Split(scanner.Text(), ",") split := strings.Split(scanner.Text(), ",")
text := scanner.Text() text := scanner.Text()
logMType <- elekLogDef.PCP elekLog.Log(PCP, log.InfoLevel, text)
logMsg <- text
totalPower := 0.0 totalPower := 0.0
for _, powerIndex := range powerIndexes { for _, powerIndex := range powerIndexes {
@ -130,10 +126,10 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
powerHistories[host].Value = power powerHistories[host].Value = power
powerHistories[host] = powerHistories[host].Next() powerHistories[host] = powerHistories[host].Next()
logMType <- elekLogDef.GENERAL elekLog.WithFields(log.Fields{
logMsg <- fmt.Sprintf("Host: %s, Power %f", "Host": indexToHost[powerIndex],
indexToHost[powerIndex], (power * pcp.RAPLUnits)) "Power": fmt.Sprintf("%f", power*pcp.RAPLUnits),
}).Log(CONSOLE, log.InfoLevel, "")
totalPower += power totalPower += power
} }
clusterPower := totalPower * pcp.RAPLUnits clusterPower := totalPower * pcp.RAPLUnits
@ -143,16 +139,16 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
clusterMean := pcp.AverageClusterPowerHistory(clusterPowerHist) clusterMean := pcp.AverageClusterPowerHistory(clusterPowerHist)
logMType <- elekLogDef.GENERAL elekLog.WithFields(log.Fields{
logMsg <- fmt.Sprintf("Total power: %f, %d Sec Avg: %f", clusterPower, clusterPowerHist.Len(), clusterMean) "Total power": fmt.Sprintf("%f %d Sec", clusterPower, clusterPowerHist.Len()),
"Avg": fmt.Sprintf("%f", clusterMean),
}).Log(CONSOLE, log.InfoLevel, "")
if clusterMean >= hiThreshold { if clusterMean >= hiThreshold {
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "Need to cap a node")
logMsg <- "Need to cap a node" elekLog.Logf(CONSOLE, log.InfoLevel, "Cap values of capped victims %v", cappedVictims)
logMType <- elekLogDef.GENERAL elekLog.Logf(CONSOLE, log.InfoLevel, "Cap values of victims to uncap %v", orderCappedVictims)
logMsg <- fmt.Sprintf("Cap values of capped victims: %v", cappedVictims)
logMType <- elekLogDef.GENERAL
logMsg <- fmt.Sprintf("Cap values of victims to uncap: %v", orderCappedVictims)
// Create statics for all victims and choose one to cap // Create statics for all victims and choose one to cap
victims := make([]pcp.Victim, 0, 8) victims := make([]pcp.Victim, 0, 8)
@ -179,11 +175,11 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
} }
// Need to cap this victim. // Need to cap this victim.
if err := rapl.Cap(victims[i].Host, "rapl", 50.0); err != nil { if err := rapl.Cap(victims[i].Host, "rapl", 50.0); err != nil {
logMType <- elekLogDef.GENERAL
logMsg <- fmt.Sprintf("Error capping host %s", victims[i].Host) elekLog.Logf(CONSOLE, log.ErrorLevel, "Error capping host %s", victims[i].Host)
} else { } else {
logMType <- elekLogDef.GENERAL
logMsg <- fmt.Sprintf("Capped host[%s] at %f", victims[i].Host, 50.0) elekLog.Logf(CONSOLE, log.InfoLevel, "Capped host[%s] at %f", victims[i].Host, 50.0)
// Keeping track of this victim and it's cap value // Keeping track of this victim and it's cap value
cappedVictims[victims[i].Host] = 50.0 cappedVictims[victims[i].Host] = 50.0
newVictimFound = true newVictimFound = true
@ -206,12 +202,11 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
if capValue > constants.LowerCapLimit { if capValue > constants.LowerCapLimit {
newCapValue := getNextCapValue(capValue, 2) newCapValue := getNextCapValue(capValue, 2)
if err := rapl.Cap(alreadyCappedHosts[i], "rapl", newCapValue); err != nil { if err := rapl.Cap(alreadyCappedHosts[i], "rapl", newCapValue); err != nil {
logMType <- elekLogDef.ERROR
logMsg <- fmt.Sprintf("Error capping host[%s]", alreadyCappedHosts[i]) elekLog.Logf(CONSOLE, log.ErrorLevel, "Error capping host %s", alreadyCappedHosts[i])
} else { } else {
// Successful cap // Successful cap
logMType <- elekLogDef.GENERAL elekLog.Logf(CONSOLE, log.InfoLevel, "Capped host[%s] at %f", alreadyCappedHosts[i], newCapValue)
logMsg <- fmt.Sprintf("Capped host[%s] at %f", alreadyCappedHosts[i], newCapValue)
// Checking whether this victim can be capped further // Checking whether this victim can be capped further
if newCapValue <= constants.LowerCapLimit { if newCapValue <= constants.LowerCapLimit {
// Deleting victim from cappedVictims. // Deleting victim from cappedVictims.
@ -234,18 +229,15 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
} }
} }
if !canCapAlreadyCappedVictim { if !canCapAlreadyCappedVictim {
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "No Victim left to cap")
logMsg <- "No Victim left to cap."
} }
} }
} else if clusterMean < loThreshold { } else if clusterMean < loThreshold {
logMType <- elekLogDef.GENERAL
logMsg <- "Need to uncap a node" elekLog.Log(CONSOLE, log.InfoLevel, "Need to uncap a node")
logMType <- elekLogDef.GENERAL elekLog.Logf(CONSOLE, log.InfoLevel, "Cap values of capped victims - %v", cappedVictims)
logMsg <- fmt.Sprintf("Cap values of capped victims: %v", cappedVictims) elekLog.Logf(CONSOLE, log.InfoLevel, "Cap values of victims to uncap - %v", orderCappedVictims)
logMType <- elekLogDef.GENERAL
logMsg <- fmt.Sprintf("Cap values of victims to uncap: %v", orderCappedVictims)
if len(orderCapped) > 0 { if len(orderCapped) > 0 {
// We pick the host that is capped the most to uncap. // We pick the host that is capped the most to uncap.
orderCappedToSort := utilities.GetPairList(orderCappedVictims) orderCappedToSort := utilities.GetPairList(orderCappedVictims)
@ -255,12 +247,11 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
// This is a floating point operation and might suffer from precision loss. // This is a floating point operation and might suffer from precision loss.
newUncapValue := orderCappedVictims[hostToUncap] * 2.0 newUncapValue := orderCappedVictims[hostToUncap] * 2.0
if err := rapl.Cap(hostToUncap, "rapl", newUncapValue); err != nil { if err := rapl.Cap(hostToUncap, "rapl", newUncapValue); err != nil {
logMType <- elekLogDef.ERROR
logMsg <- fmt.Sprintf("Error uncapping host[%s]", hostToUncap) elekLog.Logf(CONSOLE, log.ErrorLevel, "Error uncapping host %s", hostToUncap)
} else { } else {
// Successful uncap // Successful uncap
logMType <- elekLogDef.GENERAL elekLog.Logf(CONSOLE, log.InfoLevel, "Uncapped host[%s] to %f", hostToUncap, newUncapValue)
logMsg <- fmt.Sprintf("Uncapped host[%s] to %f", hostToUncap, newUncapValue)
// Can we uncap this host further. If not, then we remove its entry from orderCapped // Can we uncap this host further. If not, then we remove its entry from orderCapped
if newUncapValue >= 100.0 { // can compare using == if newUncapValue >= 100.0 { // can compare using ==
// Deleting entry from orderCapped // Deleting entry from orderCapped
@ -281,8 +272,7 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
} }
} }
} else { } else {
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "No host staged for Uncapped")
logMsg <- "No host staged for Uncapped"
} }
} }
} }
@ -291,9 +281,7 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
}(logging, hiThreshold, loThreshold) }(logging, hiThreshold, loThreshold)
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "PCP logging started")
logMsg <- "PCP logging started"
if err := cmd.Start(); err != nil { if err := cmd.Start(); err != nil {
log.Fatal(err) log.Fatal(err)
} }
@ -302,8 +290,7 @@ func StartPCPLogAndProgressiveExtremaCap(quit chan struct{}, logging *bool, hiTh
select { select {
case <-quit: case <-quit:
logMType <- elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "Stopping PCP logging in 5 seconds")
logMsg <- "Stopping PCP logging in 5 seconds"
time.Sleep(5 * time.Second) time.Sleep(5 * time.Second)
// http://stackoverflow.com/questions/22470193/why-wont-go-kill-a-child-process-correctly // http://stackoverflow.com/questions/22470193/why-wont-go-kill-a-child-process-correctly

12
rapl/cap.go
View file

@ -1,31 +1,31 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package rapl package rapl
import ( import (
"os" "os"
"strconv" "strconv"
"strings"
"github.com/pkg/errors" "github.com/pkg/errors"
elekEnv "github.com/spdfg/elektron/environment" elekEnv "github.com/spdfg/elektron/environment"
"golang.org/x/crypto/ssh" "golang.org/x/crypto/ssh"
"strings"
) )
func Cap(host, username string, percentage float64) error { func Cap(host, username string, percentage float64) error {

44
scheduler.go
View file

@ -21,7 +21,6 @@ package main // import github.com/spdfg/elektron
import ( import (
"flag" "flag"
"fmt" "fmt"
"log"
"os" "os"
"os/signal" "os/signal"
"strings" "strings"
@ -30,8 +29,10 @@ import (
"github.com/golang/protobuf/proto" "github.com/golang/protobuf/proto"
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
sched "github.com/mesos/mesos-go/api/v0/scheduler" sched "github.com/mesos/mesos-go/api/v0/scheduler"
log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/def" "github.com/spdfg/elektron/def"
elekLogDef "github.com/spdfg/elektron/logging/def" elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
"github.com/spdfg/elektron/pcp" "github.com/spdfg/elektron/pcp"
"github.com/spdfg/elektron/powerCap" "github.com/spdfg/elektron/powerCap"
"github.com/spdfg/elektron/schedulers" "github.com/spdfg/elektron/schedulers"
@ -55,6 +56,7 @@ var fixFirstSchedPol = flag.String("fixFirstSchedPol", "", "Name of the scheduli
var fixSchedWindow = flag.Bool("fixSchedWindow", false, "Fix the size of the scheduling window that every deployed scheduling policy should schedule, provided switching is enabled.") var fixSchedWindow = flag.Bool("fixSchedWindow", false, "Fix the size of the scheduling window that every deployed scheduling policy should schedule, provided switching is enabled.")
var schedWindowSize = flag.Int("schedWindowSize", 200, "Size of the scheduling window if fixSchedWindow is set.") var schedWindowSize = flag.Int("schedWindowSize", 200, "Size of the scheduling window if fixSchedWindow is set.")
var schedPolSwitchCriteria = flag.String("schedPolSwitchCriteria", "taskDist", "Scheduling policy switching criteria.") var schedPolSwitchCriteria = flag.String("schedPolSwitchCriteria", "taskDist", "Scheduling policy switching criteria.")
var logConfigFilename = flag.String("logConfigFilename", "logConfig.yaml", "Log Configuration file name")
// Short hand args // Short hand args
func init() { func init() {
@ -76,6 +78,7 @@ func init() {
flag.BoolVar(fixSchedWindow, "fixSw", false, "Fix the size of the scheduling window that every deployed scheduling policy should schedule, provided switching is enabled (shorthand).") flag.BoolVar(fixSchedWindow, "fixSw", false, "Fix the size of the scheduling window that every deployed scheduling policy should schedule, provided switching is enabled (shorthand).")
flag.IntVar(schedWindowSize, "swSize", 200, "Size of the scheduling window if fixSchedWindow is set (shorthand).") flag.IntVar(schedWindowSize, "swSize", 200, "Size of the scheduling window if fixSchedWindow is set (shorthand).")
flag.StringVar(schedPolSwitchCriteria, "spsCriteria", "taskDist", "Scheduling policy switching criteria (shorthand).") flag.StringVar(schedPolSwitchCriteria, "spsCriteria", "taskDist", "Scheduling policy switching criteria (shorthand).")
flag.StringVar(logConfigFilename, "lgCfg", "logConfig.yaml", "Log Configuration file name (shorthand).")
} }
func listAllSchedulingPolicies() { func listAllSchedulingPolicies() {
@ -95,10 +98,6 @@ func main() {
os.Exit(1) os.Exit(1)
} }
// Logging channels.
logMType := make(chan elekLogDef.LogMessageType)
logMsg := make(chan string)
// First we need to build the scheduler using scheduler options. // First we need to build the scheduler using scheduler options.
var schedOptions []schedulers.SchedulerOptions = make([]schedulers.SchedulerOptions, 0, 10) var schedOptions []schedulers.SchedulerOptions = make([]schedulers.SchedulerOptions, 0, 10)
@ -120,11 +119,6 @@ func main() {
pcpLog := make(chan struct{}) pcpLog := make(chan struct{})
recordPCP := false recordPCP := false
// Logging channels.
// These channels are used by the framework to log messages.
// The channels are used to send the type of log message and the message string.
schedOptions = append(schedOptions, schedulers.WithLoggingChannels(logMType, logMsg))
// Shutdown indicator channels. // Shutdown indicator channels.
// These channels are used to notify, // These channels are used to notify,
// 1. scheduling is complete. // 1. scheduling is complete.
@ -228,28 +222,25 @@ func main() {
log.Fatal(fmt.Sprintf("Unable to create scheduler driver: %s", err)) log.Fatal(fmt.Sprintf("Unable to create scheduler driver: %s", err))
} }
// If here, then all command-line arguments validate.
// Creating logger and attaching different logging platforms.
startTime := time.Now()
formattedStartTime := startTime.Format("20060102150405")
// Checking if prefix contains any special characters. // Checking if prefix contains any special characters.
if strings.Contains(*pcplogPrefix, "/") { if strings.Contains(*pcplogPrefix, "/") {
log.Fatal("log file prefix should not contain '/'.") log.Fatal("log file prefix should not contain '/'.")
} }
logPrefix := *pcplogPrefix + "_" + formattedStartTime
logger := elekLogDef.BuildLogger(startTime, logPrefix) // Build Logger.
// Starting the logging go-routine. if err := elekLog.BuildLogger(*pcplogPrefix, *logConfigFilename); err != nil {
go logger.Listen(logMType, logMsg) log.Fatal(err)
}
// Starting PCP logging. // Starting PCP logging.
if noPowercap { if noPowercap {
go pcp.Start(pcpLog, &recordPCP, logMType, logMsg, *pcpConfigFile) go pcp.Start(pcpLog, &recordPCP, *pcpConfigFile)
} else if extrema { } else if extrema {
go powerCap.StartPCPLogAndExtremaDynamicCap(pcpLog, &recordPCP, *hiThreshold, go powerCap.StartPCPLogAndExtremaDynamicCap(pcpLog, &recordPCP, *hiThreshold,
*loThreshold, logMType, logMsg, *pcpConfigFile) *loThreshold, *pcpConfigFile)
} else if progExtrema { } else if progExtrema {
go powerCap.StartPCPLogAndProgressiveExtremaCap(pcpLog, &recordPCP, *hiThreshold, go powerCap.StartPCPLogAndProgressiveExtremaCap(pcpLog, &recordPCP, *hiThreshold,
*loThreshold, logMType, logMsg, *pcpConfigFile) *loThreshold, *pcpConfigFile)
} }
// Take a second between starting PCP log and continuing. // Take a second between starting PCP log and continuing.
@ -284,8 +275,6 @@ func main() {
close(pcpLog) close(pcpLog)
time.Sleep(5 * time.Second) //Wait for PCP to log a few more seconds time.Sleep(5 * time.Second) //Wait for PCP to log a few more seconds
// Closing logging channels. // Closing logging channels.
close(logMType)
close(logMsg)
//case <-time.After(shutdownTimeout): //case <-time.After(shutdownTimeout):
} }
@ -296,7 +285,10 @@ func main() {
// Starting the scheduler driver. // Starting the scheduler driver.
if status, err := driver.Run(); err != nil { if status, err := driver.Run(); err != nil {
log.Printf("Framework stopped with status %s and error: %s\n", status.String(), err.Error()) elekLog.WithFields(log.Fields{
"status": status.String(),
"error": err.Error(),
}).Log(CONSOLE, log.ErrorLevel, "Framework stopped ")
} }
log.Println("Exiting...") elekLog.Log(CONSOLE, log.InfoLevel, "Exiting...")
} }

13
schedulers/MaxGreedyMins.go
View file

@ -1,28 +1,27 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedulers package schedulers
import ( import (
"log"
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
sched "github.com/mesos/mesos-go/api/v0/scheduler" sched "github.com/mesos/mesos-go/api/v0/scheduler"
log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/def" "github.com/spdfg/elektron/def"
"github.com/spdfg/elektron/utilities/mesosUtils" "github.com/spdfg/elektron/utilities/mesosUtils"
"github.com/spdfg/elektron/utilities/offerUtils" "github.com/spdfg/elektron/utilities/offerUtils"

13
schedulers/MaxMin.go
View file

@ -1,28 +1,27 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedulers package schedulers
import ( import (
"log"
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
sched "github.com/mesos/mesos-go/api/v0/scheduler" sched "github.com/mesos/mesos-go/api/v0/scheduler"
log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/def" "github.com/spdfg/elektron/def"
"github.com/spdfg/elektron/utilities/mesosUtils" "github.com/spdfg/elektron/utilities/mesosUtils"
"github.com/spdfg/elektron/utilities/offerUtils" "github.com/spdfg/elektron/utilities/offerUtils"

166
schedulers/base.go
View file

@ -1,27 +1,26 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedulers package schedulers
import ( import (
"bytes" "bytes"
"fmt" "fmt"
"log"
"sync" "sync"
"time" "time"
@ -29,8 +28,10 @@ import (
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
"github.com/mesos/mesos-go/api/v0/mesosutil" "github.com/mesos/mesos-go/api/v0/mesosutil"
sched "github.com/mesos/mesos-go/api/v0/scheduler" sched "github.com/mesos/mesos-go/api/v0/scheduler"
log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/def" "github.com/spdfg/elektron/def"
elekLogDef "github.com/spdfg/elektron/logging/def" elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
"github.com/spdfg/elektron/utilities" "github.com/spdfg/elektron/utilities"
"github.com/spdfg/elektron/utilities/schedUtils" "github.com/spdfg/elektron/utilities/schedUtils"
) )
@ -68,11 +69,6 @@ type BaseScheduler struct {
schedTrace *log.Logger schedTrace *log.Logger
// Send the type of the message to be logged
logMsgType chan elekLogDef.LogMessageType
// Send the message to be logged
logMsg chan string
mutex sync.Mutex mutex sync.Mutex
// Whether switching of scheduling policies at runtime has been enabled // Whether switching of scheduling policies at runtime has been enabled
@ -252,179 +248,126 @@ func (s *BaseScheduler) StatusUpdate(driver sched.SchedulerDriver, status *mesos
} }
} }
func (s *BaseScheduler) Log(lmt elekLogDef.LogMessageType, msg string) {
s.mutex.Lock()
s.logMsgType <- lmt
s.logMsg <- msg
s.mutex.Unlock()
}
func (s *BaseScheduler) LogTaskStarting(ts *def.Task, offer *mesos.Offer) { func (s *BaseScheduler) LogTaskStarting(ts *def.Task, offer *mesos.Offer) {
lmt := elekLogDef.GENERAL
msgColor := elekLogDef.LogMessageColors[lmt]
var msg string
if ts == nil { if ts == nil {
msg = msgColor.Sprintf("TASKS STARTING... host = [%s]", offer.GetHostname()) elekLog.WithField("host", offer.GetHostname()).Log(CONSOLE, log.InfoLevel, "TASKS STARTING...")
} else { } else {
msg = msgColor.Sprintf("TASK STARTING... task = [%s], Instance = %d, host = [%s]", elekLog.WithFields(log.Fields{
ts.Name, *ts.Instances, offer.GetHostname()) "task": ts.Name,
"Instance": fmt.Sprintf("%d", *ts.Instances),
"host": offer.GetHostname(),
}).Log(CONSOLE, log.InfoLevel, "TASK STARTING... ")
} }
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogTaskWattsConsideration(ts def.Task, host string, wattsToConsider float64) { func (s *BaseScheduler) LogTaskWattsConsideration(ts def.Task, host string, wattsToConsider float64) {
lmt := elekLogDef.GENERAL elekLog.WithFields(log.Fields{
msgColor := elekLogDef.LogMessageColors[lmt] "task": ts.Name,
msg := msgColor.Sprintf("Watts considered for task[%s] and host[%s] = %f Watts", "host": host,
ts.Name, host, wattsToConsider) "Watts": fmt.Sprintf("%f", wattsToConsider),
s.Log(lmt, msg) }).Log(CONSOLE, log.InfoLevel, "Watts considered for ")
} }
func (s *BaseScheduler) LogOffersReceived(offers []*mesos.Offer) { func (s *BaseScheduler) LogOffersReceived(offers []*mesos.Offer) {
lmt := elekLogDef.GENERAL elekLog.WithField("numOffers", fmt.Sprintf("%d", len(offers))).Log(CONSOLE, log.InfoLevel, "Resource offers received")
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprintf("Received %d resource offers", len(offers))
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogNoPendingTasksDeclineOffers(offer *mesos.Offer) { func (s *BaseScheduler) LogNoPendingTasksDeclineOffers(offer *mesos.Offer) {
lmt := elekLogDef.WARNING elekLog.Logf(CONSOLE, log.WarnLevel, "DECLINING OFFER for host %s. No tasks left to schedule", offer.GetHostname())
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprintf("DECLINING OFFER for host[%s]... "+
"No tasks left to schedule", offer.GetHostname())
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogNumberOfRunningTasks() { func (s *BaseScheduler) LogNumberOfRunningTasks() {
lmt := elekLogDef.GENERAL elekLog.Logf(CONSOLE, log.InfoLevel, "Number of tasks still running %d", s.tasksRunning)
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprintf("Number of tasks still Running = %d", s.tasksRunning)
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogCoLocatedTasks(slaveID string) { func (s *BaseScheduler) LogCoLocatedTasks(slaveID string) {
lmt := elekLogDef.GENERAL
msgColor := elekLogDef.LogMessageColors[lmt]
buffer := bytes.Buffer{} buffer := bytes.Buffer{}
buffer.WriteString(fmt.Sprintln("Colocated with:"))
s.TasksRunningMutex.Lock() s.TasksRunningMutex.Lock()
for taskName := range s.Running[slaveID] { for taskName := range s.Running[slaveID] {
buffer.WriteString(fmt.Sprintln(taskName)) buffer.WriteString(fmt.Sprintln(taskName))
} }
s.TasksRunningMutex.Unlock() s.TasksRunningMutex.Unlock()
msg := msgColor.Sprintf(buffer.String()) elekLog.WithField("Tasks", buffer.String()).Log(CONSOLE, log.InfoLevel, "Colocated with")
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogSchedTrace(taskToSchedule *mesos.TaskInfo, offer *mesos.Offer) { func (s *BaseScheduler) LogSchedTrace(taskToSchedule *mesos.TaskInfo, offer *mesos.Offer) {
msg := fmt.Sprint(offer.GetHostname() + ":" + taskToSchedule.GetTaskId().GetValue()) elekLog.WithField(offer.GetHostname(), taskToSchedule.GetTaskId().GetValue()).Log(SCHED_TRACE, log.InfoLevel, "")
s.Log(elekLogDef.SCHED_TRACE, msg)
} }
func (s *BaseScheduler) LogTerminateScheduler() { func (s *BaseScheduler) LogTerminateScheduler() {
lmt := elekLogDef.GENERAL elekLog.Log(CONSOLE, log.InfoLevel, "Done scheduling all tasks!")
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprint("Done scheduling all tasks!")
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogInsufficientResourcesDeclineOffer(offer *mesos.Offer, func (s *BaseScheduler) LogInsufficientResourcesDeclineOffer(offer *mesos.Offer,
offerResources ...interface{}) { offerResources ...interface{}) {
lmt := elekLogDef.WARNING
msgColor := elekLogDef.LogMessageColors[lmt]
buffer := bytes.Buffer{} buffer := bytes.Buffer{}
buffer.WriteString(fmt.Sprintln("DECLINING OFFER... Offer has insufficient resources to launch a task")) buffer.WriteString(fmt.Sprintf("<CPU: %f, RAM: %f, Watts: %f>", offerResources...))
buffer.WriteString(fmt.Sprintf("Offer Resources <CPU: %f, RAM: %f, Watts: %f>", offerResources...)) elekLog.WithField("Offer Resources", buffer.String()).Log(CONSOLE,
msg := msgColor.Sprint(buffer.String()) log.WarnLevel, "DECLINING OFFER... Offer has insufficient resources to launch a task")
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogOfferRescinded(offerID *mesos.OfferID) { func (s *BaseScheduler) LogOfferRescinded(offerID *mesos.OfferID) {
lmt := elekLogDef.ERROR elekLog.WithField("OfferID", *offerID.Value).Log(CONSOLE, log.ErrorLevel, "OFFER RESCINDED")
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprintf("OFFER RESCINDED: OfferID = %s", offerID)
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogSlaveLost(slaveID *mesos.SlaveID) { func (s *BaseScheduler) LogSlaveLost(slaveID *mesos.SlaveID) {
lmt := elekLogDef.ERROR elekLog.WithField("SlaveID", *slaveID.Value).Log(CONSOLE, log.ErrorLevel, "SLAVE LOST")
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprintf("SLAVE LOST: SlaveID = %s", slaveID)
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogExecutorLost(executorID *mesos.ExecutorID, slaveID *mesos.SlaveID) { func (s *BaseScheduler) LogExecutorLost(executorID *mesos.ExecutorID, slaveID *mesos.SlaveID) {
lmt := elekLogDef.ERROR elekLog.WithFields(log.Fields{
msgColor := elekLogDef.LogMessageColors[lmt] "ExecutorID": *executorID.Value,
msg := msgColor.Sprintf("EXECUTOR LOST: ExecutorID = %s, SlaveID = %s", executorID, slaveID) "SlaveID": *slaveID.Value,
s.Log(lmt, msg) }).Log(CONSOLE, log.ErrorLevel, "EXECUTOR LOST")
} }
func (s *BaseScheduler) LogFrameworkMessage(executorID *mesos.ExecutorID, func (s *BaseScheduler) LogFrameworkMessage(executorID *mesos.ExecutorID,
slaveID *mesos.SlaveID, message string) { slaveID *mesos.SlaveID, message string) {
lmt := elekLogDef.GENERAL elekLog.Logf(CONSOLE, log.InfoLevel, "Received Framework message from executor %v", executorID)
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprintf("Received Framework message from executor [%s]: %s", executorID, message)
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogMesosError(err string) { func (s *BaseScheduler) LogMesosError(err string) {
lmt := elekLogDef.ERROR elekLog.Logf(CONSOLE, log.ErrorLevel, "MESOS CONSOLE %v", err)
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprintf("MESOS ERROR: %s", err)
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogElectronError(err error) { func (s *BaseScheduler) LogElectronError(err error) {
lmt := elekLogDef.ERROR elekLog.Logf(CONSOLE, log.ErrorLevel, "ELEKTRON CONSOLE %v", err)
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprintf("ELECTRON ERROR: %v", err)
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogFrameworkRegistered(frameworkID *mesos.FrameworkID, func (s *BaseScheduler) LogFrameworkRegistered(frameworkID *mesos.FrameworkID,
masterInfo *mesos.MasterInfo) { masterInfo *mesos.MasterInfo) {
lmt := elekLogDef.SUCCESS elekLog.WithFields(log.Fields{
msgColor := elekLogDef.LogMessageColors[lmt] "frameworkID": *frameworkID.Value,
msg := msgColor.Sprintf("FRAMEWORK REGISTERED! frameworkID = %s, master = %s", "master": fmt.Sprintf("%v", masterInfo),
frameworkID, masterInfo) }).Log(CONSOLE, log.InfoLevel, "FRAMEWORK REGISTERED!")
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogFrameworkReregistered(masterInfo *mesos.MasterInfo) { func (s *BaseScheduler) LogFrameworkReregistered(masterInfo *mesos.MasterInfo) {
lmt := elekLogDef.GENERAL elekLog.WithField("master", fmt.Sprintf("%v", masterInfo)).Log(CONSOLE, log.InfoLevel, "Framework re-registered")
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprintf("Framework re-registered with master %s", masterInfo)
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogDisconnected() { func (s *BaseScheduler) LogDisconnected() {
lmt := elekLogDef.WARNING elekLog.Log(CONSOLE, log.WarnLevel, "Framework disconnected with master")
msgColor := elekLogDef.LogMessageColors[lmt]
msg := msgColor.Sprint("Framework disconnected with master")
s.Log(lmt, msg)
} }
func (s *BaseScheduler) LogTaskStatusUpdate(status *mesos.TaskStatus) { func (s *BaseScheduler) LogTaskStatusUpdate(status *mesos.TaskStatus) {
var lmt elekLogDef.LogMessageType level := log.InfoLevel
switch *status.State { switch *status.State {
case mesos.TaskState_TASK_ERROR, mesos.TaskState_TASK_FAILED, case mesos.TaskState_TASK_ERROR, mesos.TaskState_TASK_FAILED,
mesos.TaskState_TASK_KILLED, mesos.TaskState_TASK_LOST: mesos.TaskState_TASK_KILLED, mesos.TaskState_TASK_LOST:
lmt = elekLogDef.ERROR level = log.ErrorLevel
case mesos.TaskState_TASK_FINISHED:
lmt = elekLogDef.SUCCESS
default: default:
lmt = elekLogDef.GENERAL level = log.InfoLevel
} }
msgColor := elekLogDef.LogMessageColors[lmt] elekLog.WithFields(log.Fields{
msg := elekLogDef.LogMessageColors[elekLogDef.GENERAL].Sprintf("Task Status received for task [%s] --> %s", "task": *status.TaskId.Value,
*status.TaskId.Value, msgColor.Sprint(NameFor(status.State))) "state": NameFor(status.State),
s.Log(lmt, msg) }).Log(CONSOLE, level, "Task Status received")
} }
func (s *BaseScheduler) LogSchedPolicySwitch(name string, nextPolicy SchedPolicyState) { func (s *BaseScheduler) LogSchedPolicySwitch(name string, nextPolicy SchedPolicyState) {
logSPS := func() { logSPS := func() {
s.Log(elekLogDef.SPS, name) elekLog.WithField("Name", name).Log(SPS, log.InfoLevel, "")
} }
if s.hasReceivedResourceOffers && (s.curSchedPolicy != nextPolicy) { if s.hasReceivedResourceOffers && (s.curSchedPolicy != nextPolicy) {
logSPS() logSPS()
@ -433,10 +376,13 @@ func (s *BaseScheduler) LogSchedPolicySwitch(name string, nextPolicy SchedPolicy
} }
// Logging the size of the scheduling window and the scheduling policy // Logging the size of the scheduling window and the scheduling policy
// that is going to schedule the tasks in the scheduling window. // that is going to schedule the tasks in the scheduling window.
s.Log(elekLogDef.SCHED_WINDOW, fmt.Sprintf("%d %s", s.schedWindowSize, name)) elekLog.WithFields(log.Fields{
"Window size": fmt.Sprintf("%d", s.schedWindowSize),
"Name": name,
}).Log(SCHED_WINDOW, log.InfoLevel, "")
} }
func (s *BaseScheduler) LogClsfnAndTaskDistOverhead(overhead time.Duration) { func (s *BaseScheduler) LogClsfnAndTaskDistOverhead(overhead time.Duration) {
// Logging the overhead in microseconds. // Logging the overhead in microseconds.
s.Log(elekLogDef.CLSFN_TASKDIST_OVERHEAD, fmt.Sprintf("%f", float64(overhead.Nanoseconds())/1000.0)) elekLog.WithField("Overhead in microseconds", fmt.Sprintf("%f", float64(overhead.Nanoseconds())/1000.0)).Log(CLSFN_TASKDISTR_OVERHEAD, log.InfoLevel, "")
} }

13
schedulers/bin-packing.go
View file

@ -1,28 +1,27 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedulers package schedulers
import ( import (
"log"
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
sched "github.com/mesos/mesos-go/api/v0/scheduler" sched "github.com/mesos/mesos-go/api/v0/scheduler"
log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/def" "github.com/spdfg/elektron/def"
"github.com/spdfg/elektron/utilities/mesosUtils" "github.com/spdfg/elektron/utilities/mesosUtils"
"github.com/spdfg/elektron/utilities/offerUtils" "github.com/spdfg/elektron/utilities/offerUtils"

13
schedulers/electronScheduler.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedulers package schedulers
@ -24,7 +24,6 @@ import (
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
sched "github.com/mesos/mesos-go/api/v0/scheduler" sched "github.com/mesos/mesos-go/api/v0/scheduler"
"github.com/spdfg/elektron/def" "github.com/spdfg/elektron/def"
elekLogDef "github.com/spdfg/elektron/logging/def"
) )
// Implements mesos scheduler. // Implements mesos scheduler.
@ -38,8 +37,6 @@ type ElectronScheduler interface {
// Each of these functions are supposed to call the Log(...) that sends the // Each of these functions are supposed to call the Log(...) that sends the
// log message type, and the log message to the corresponding channels. // log message type, and the log message to the corresponding channels.
// Pass the logMessageType and the logMessage to the loggers for logging.
Log(logMType elekLogDef.LogMessageType, logMsg string)
// To be called when about to launch a task. // To be called when about to launch a task.
// Log message indicating that a task is about to start executing. // Log message indicating that a task is about to start executing.
// Also, log the host on which the task is going to be launched. // Also, log the host on which the task is going to be launched.

10
schedulers/first-fit.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedulers package schedulers

20
schedulers/helpers.go
View file

@ -20,24 +20,26 @@ package schedulers
import ( import (
"fmt" "fmt"
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
sched "github.com/mesos/mesos-go/api/v0/scheduler" sched "github.com/mesos/mesos-go/api/v0/scheduler"
"github.com/pkg/errors" "github.com/pkg/errors"
log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/constants" "github.com/spdfg/elektron/constants"
"github.com/spdfg/elektron/def" "github.com/spdfg/elektron/def"
elekLogDef "github.com/spdfg/elektron/logging/def" elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
"github.com/spdfg/elektron/utilities" "github.com/spdfg/elektron/utilities"
"github.com/spdfg/elektron/utilities/mesosUtils" "github.com/spdfg/elektron/utilities/mesosUtils"
"log"
) )
func coLocated(tasks map[string]bool, s BaseScheduler) { func coLocated(tasks map[string]bool, s BaseScheduler) {
for task := range tasks { for _, task := range tasks {
s.Log(elekLogDef.GENERAL, task) elekLog.WithField("Task", fmt.Sprintf("%v", task)).Log(CONSOLE, log.InfoLevel, "")
} }
s.Log(elekLogDef.GENERAL, "---------------------") elekLog.Log(CONSOLE, log.InfoLevel, "---------------------")
} }
// Get the powerClass of the given hostname. // Get the powerClass of the given hostname.
@ -129,14 +131,6 @@ func WithPCPLog(pcpLog chan struct{}) SchedulerOptions {
} }
} }
func WithLoggingChannels(lmt chan elekLogDef.LogMessageType, msg chan string) SchedulerOptions {
return func(s ElectronScheduler) error {
s.(*BaseScheduler).logMsgType = lmt
s.(*BaseScheduler).logMsg = msg
return nil
}
}
func WithSchedPolSwitchEnabled(enableSchedPolicySwitch bool, switchingCriteria string) SchedulerOptions { func WithSchedPolSwitchEnabled(enableSchedPolicySwitch bool, switchingCriteria string) SchedulerOptions {
return func(s ElectronScheduler) error { return func(s ElectronScheduler) error {
s.(*BaseScheduler).schedPolSwitchEnabled = enableSchedPolicySwitch s.(*BaseScheduler).schedPolSwitchEnabled = enableSchedPolicySwitch

33
schedulers/schedPolicy.go
View file

@ -1,32 +1,33 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedulers package schedulers
import ( import (
"fmt" "fmt"
"log"
"time" "time"
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
sched "github.com/mesos/mesos-go/api/v0/scheduler" sched "github.com/mesos/mesos-go/api/v0/scheduler"
log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/def" "github.com/spdfg/elektron/def"
elekLogDef "github.com/spdfg/elektron/logging/def" elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
) )
type SchedPolicyContext interface { type SchedPolicyContext interface {
@ -89,7 +90,7 @@ func switchTaskDistBased(baseSchedRef *BaseScheduler) string {
// Determine the distribution of tasks in the new scheduling window. // Determine the distribution of tasks in the new scheduling window.
taskDist, err := def.GetTaskDistributionInWindow(baseSchedRef.schedWindowSize, baseSchedRef.tasks) taskDist, err := def.GetTaskDistributionInWindow(baseSchedRef.schedWindowSize, baseSchedRef.tasks)
baseSchedRef.LogClsfnAndTaskDistOverhead(time.Now().Sub(startTime)) baseSchedRef.LogClsfnAndTaskDistOverhead(time.Now().Sub(startTime))
baseSchedRef.Log(elekLogDef.GENERAL, fmt.Sprintf("Switching... TaskDistribution[%f]", taskDist)) elekLog.WithField("Task Distribution", fmt.Sprintf("%f", taskDist)).Log(CONSOLE, log.InfoLevel, "Switching... ")
if err != nil { if err != nil {
// All the tasks in the window were only classified into 1 cluster. // All the tasks in the window were only classified into 1 cluster.
// Max-Min and Max-GreedyMins would work the same way as Bin-Packing for this situation. // Max-Min and Max-GreedyMins would work the same way as Bin-Packing for this situation.
@ -217,9 +218,11 @@ func (bsps *baseSchedPolicyState) SwitchIfNecessary(spc SchedPolicyContext) {
switchToPolicyName = switchBasedOn[baseSchedRef.schedPolSwitchCriteria](baseSchedRef) switchToPolicyName = switchBasedOn[baseSchedRef.schedPolSwitchCriteria](baseSchedRef)
} else { } else {
// We continue working with the currently deployed scheduling policy. // We continue working with the currently deployed scheduling policy.
log.Println("Continuing with the current scheduling policy...") elekLog.Log(CONSOLE, log.InfoLevel, "Continuing with the current scheduling policy...")
log.Printf("TasksScheduled[%d], SchedWindowSize[%d]", bsps.numTasksScheduled, elekLog.WithFields(log.Fields{
baseSchedRef.schedWindowSize) "TasksScheduled": fmt.Sprintf("%d", bsps.numTasksScheduled),
"SchedWindowSize": fmt.Sprintf("%d", baseSchedRef.schedWindowSize),
}).Log(CONSOLE, log.InfoLevel, "")
return return
} }
} }
@ -231,9 +234,11 @@ func (bsps *baseSchedPolicyState) SwitchIfNecessary(spc SchedPolicyContext) {
bsps.numTasksScheduled = 0 bsps.numTasksScheduled = 0
} else { } else {
// We continue working with the currently deployed scheduling policy. // We continue working with the currently deployed scheduling policy.
log.Println("Continuing with the current scheduling policy...") elekLog.Log(CONSOLE, log.InfoLevel, "Continuing with the current scheduling policy...")
log.Printf("TasksScheduled[%d], SchedWindowSize[%d]", bsps.numTasksScheduled, elekLog.WithFields(log.Fields{
baseSchedRef.schedWindowSize) "TasksScheduled": fmt.Sprintf("%d", bsps.numTasksScheduled),
"SchedWindowSize": fmt.Sprintf("%d", baseSchedRef.schedWindowSize),
}).Log(CONSOLE, log.InfoLevel, "")
return return
} }
} }

10
schedulers/states.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedulers package schedulers

10
schedulers/store.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedulers package schedulers

10
utilities/mesosUtils/mesosUtils.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package mesosUtils package mesosUtils

21
utilities/offerUtils/offerUtils.go
View file

@ -1,29 +1,31 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package offerUtils package offerUtils
import ( import (
"log"
"strings" "strings"
mesos "github.com/mesos/mesos-go/api/v0/mesosproto" mesos "github.com/mesos/mesos-go/api/v0/mesosproto"
log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/constants" "github.com/spdfg/elektron/constants"
elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
) )
func OfferAgg(offer *mesos.Offer) (float64, float64, float64) { func OfferAgg(offer *mesos.Offer) (float64, float64, float64) {
@ -88,12 +90,15 @@ func UpdateEnvironment(offer *mesos.Offer) {
var host = offer.GetHostname() var host = offer.GetHostname()
// If this host is not present in the set of hosts. // If this host is not present in the set of hosts.
if _, ok := constants.Hosts[host]; !ok { if _, ok := constants.Hosts[host]; !ok {
log.Printf("New host detected. Adding host [%s]", host) elekLog.WithField("host", host).Log(CONSOLE, log.InfoLevel, "New host detected")
// Add this host. // Add this host.
constants.Hosts[host] = struct{}{} constants.Hosts[host] = struct{}{}
// Get the power class of this host. // Get the power class of this host.
class := PowerClass(offer) class := PowerClass(offer)
log.Printf("Registering the power class... Host [%s] --> PowerClass [%s]", host, class) elekLog.WithFields(log.Fields{
"host": host,
"PowerClass": class,
}).Log(CONSOLE, log.InfoLevel, "Registering the power class...")
// If new power class, register the power class. // If new power class, register the power class.
if _, ok := constants.PowerClasses[class]; !ok { if _, ok := constants.PowerClasses[class]; !ok {
constants.PowerClasses[class] = make(map[string]struct{}) constants.PowerClasses[class] = make(map[string]struct{})

10
utilities/runAvg/runAvg.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
/* /*
A utility to calculate the running average. A utility to calculate the running average.

19
utilities/schedUtils/schedUtils.go
View file

@ -1,27 +1,28 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package schedUtils package schedUtils
import ( import (
"log" log "github.com/sirupsen/logrus"
"github.com/spdfg/elektron/def" "github.com/spdfg/elektron/def"
elekLog "github.com/spdfg/elektron/logging"
. "github.com/spdfg/elektron/logging/types"
"github.com/spdfg/elektron/utilities" "github.com/spdfg/elektron/utilities"
) )
@ -77,8 +78,8 @@ func (s *fillNextOfferCycle) apply(taskQueue []def.Task) (int, int) {
for _, task := range taskQueue { for _, task := range taskQueue {
numberOfTasksTraversed++ numberOfTasksTraversed++
for i := *task.Instances; i > 0; i-- { for i := *task.Instances; i > 0; i-- {
log.Printf("Checking if Instance #%d of Task[%s] can be scheduled "+ elekLog.Logf(CONSOLE, log.InfoLevel,
"during the next offer cycle...", i, task.Name) "Checking if Instance #%d of Task[%s] can be scheduled "+"during the next offer cycle...", i, task.Name)
if canSchedule(task) { if canSchedule(task) {
filledCPU += task.CPU filledCPU += task.CPU
filledRAM += task.RAM filledRAM += task.RAM

10
utilities/trackResourceUsage.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package utilities package utilities

10
utilities/utils.go
View file

@ -1,20 +1,20 @@
// Copyright (C) 2018 spdfg // Copyright (C) 2018 spdfg
// //
// This file is part of Elektron. // This file is part of Elektron.
// //
// Elektron is free software: you can redistribute it and/or modify // Elektron is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by // it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or // the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version. // (at your option) any later version.
// //
// Elektron is distributed in the hope that it will be useful, // Elektron is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of // but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details. // GNU General Public License for more details.
// //
// You should have received a copy of the GNU General Public License // You should have received a copy of the GNU General Public License
// along with Elektron. If not, see <http://www.gnu.org/licenses/>. // along with Elektron. If not, see <http://www.gnu.org/licenses/>.
// //
package utilities package utilities