4.5. Items¶

The ML Items Plugin generates a JSON using the information from the configuration files that will assist in generating Lucy’s UI in Simlytiks.

These Config Files are located in Lucy/bin/models/configuration/ml/. They contain the available commands for Lucy along with some additional information that is required for the communication between Simlytiks and Lucy.

Primary Arguments¶

Main specifications for the Items task.

Argument	Detail
-output	PATH. File path to where the Items JSON will be saved. This JSON contains the information necessary for Lucy and Simlytiks to communicate.

Example¶

There is only one thing to do here: produce the Items JSON and save it so that it can be used. The information used is from the Config Files. These files must be adjusted inorder to update the Items JSON. The path at which the Items JSON is saved is specified using -output.

python bin/lucy.egg plugins ml_tasks -input ml_tasks.json -output C:/items.json

The general format of the items.json with comments:

{
   "date": "__",  // help determine if item.json is up-to-date
   "version": "__",  // help determine if item.json is up-to-date
   "items": [  // possible Lucy tasks
      "label": "Feature Importance",
      "description": "Provides scores for how each input relates to the target.", // Description of the Lucy task
      "id": "feature_importance"
      "ml_type": "explore",
      "mapsTo": "feature_importance true",  // argument used by Lucy to specify the task
      "viz_type": "barchart-horizontal",  // visualization to be used by Simlytiks
      "fields": [  // additional arguments for a task that can be adjusted by a user
         {
            "label": "Target Column"  // More informative/official name to be displayed in Simlytiks
            "description": "The name of the target column.",  // field description displayed in Simlytiks
            "name": "clabel",  // how Lucy refers to the field (often equivalent to "id")
            "id": "clabel",  // how Simlytiks refers to the field (often equivalent to "name")
            "mapsTo": "clabel",  // argument used by Lucy in conjunction with "value"
            "source": "data",  // where will the "value" come from (data or custom)
            "dataTypes": [  // datatype of "value" will likely always be one element
               "text"
            ],
            "group": "ml",  // helps Simlytiks identify this field is from Lucy
            "optional": false,  // Is this argument optional for the task it belongs to?
         },

         // more fields for a task

      ],

      // more tasks supported by Lucy

   ]

}

Predict example:

{
   "date": "__",  // help determine if item.json is up-to-date
   "version": "__",  // help determine if item.json is up-to-date
   "items": [  // possible Lucy tasks
      "label": "Predict",
      "description": "Prediction using a previously saved learning (mfile)",
      "id": "predict"
      "ml_type": "predict",
      "mapsTo": "predict",
      "viz_type": "predict",
      "fields": [
         {
            "label": "MFILE",
            "description": "This points to the path where the mfile is located",
            "name": "mfile",
            "id": "mfile",
            "mapsTo": "mfile",
            "dataTypes": [ "text" ],
            "group": "ml",
            "optional": false,
         },
         {
            "label": "data",
            "description": "Data to predict. Can be a file path or a list of dictionaries as present during learning",
            "name": "data",
            "id": "data",
            "mapsTo": "input",
            "dataTypes": [ "text" ],
            "group": "ml",
            "optional": false,
         },
         {
            "label": "work_dir",
            "description": "Working directory used in the previous learning. This will be used to find the mfile",
            "name": "work_dir",
            "id": "work_dir",
            "mapsTo": "work_dir",
            "dataTypes": [ "text" ],
            "group": "ml",
            "optional": false,
         }

      ]

   ]

}

View a complete version of the Prediction files: predit_task.json predict_output.json

View a complete version of the Items: items.json

Additional Notes: Updating the Items JSON¶

The goal to keep in mind is that it is desirable to only need to update Lucy/bin/models/configuration/ml/ configuration files to minimize duplicated work.

Suppose changes have been made to the ML plugins and things have been added correctly to the appropriate ML Config Files, then an updated Items JSON is likely necessary for Simlytiks to reflect those additions. The Items Example shows all that should be required to produce a new Items JSON–there is no need to change any code in ml_items_plugin.py.

In general, the Keys come from ml_items_plugin.py and their values come from the Config Files.

Note

Changes to the ml_items_plugin.py may be required if:

names of Keys change
new Keys are required
the JSON format changes
Items from new plugins are required (Pulls from Explore and Learn, June 2022)

Adding Items¶

It is important to make the distinction between the Config Files’ “command_line_arguments” and “supported_simlytiks_tasks”.

command_line_arguments:

arguments that the plugins uses
printed to the console when using -h

supported_simlytiks_tasks:

mirror many aspects of command_line_arguments
pulls information from command_line_arguments
used for the Items JSON

The reason for this separation was because Simlytiks required more information for some tasks but not all. Since the Items Plugin pulls information from supported_simlytiks_tasks, that will be the focus here.

Keys required for supported_simlytiks_tasks:

id - task reference for Simlytiks
label - name of the task displayed by Simlytiks
description - description of the task displayed by Simlytiks
ml_type - what plugin the task is from
mapsTo - space (” “) separated arg and option of the Lucy command to specify the task
viz_type - visual Simlytiks is to use (outdated?),
fields - arguments tied to the task whose options can be changed by the user in Simlytiks.

Adding Fields¶

The “fields” are supported arguments for Lucy. These are contained within the command_line_arguments of the Config Files. The fields in the Config Files contain requirements for a Lucy argument and its inclusion in the Items JSON.

Keys required for fields:

name - Used by Lucy to form of the command line argument. Often preceded by “-” to indicate it being an arg.
dest - Used by Lucy and is how the arg is recognized in python. Often just replaces preceding “-” with “” and other “-“s with “_”
“action”: “store_true” - Optional. Used by Lucy when just the presence of the argument is enough to trigger the

task

group - Optional. Used by Lucy when organizing the help menu print
help - Used by Lucy. Is the message printed for more information on the arguments use case
id - Used by Items. How Simlytiks refers to the field
label - Used by Items. More informative/official name to be displayed in Simlytiks
description - Used by Items. Arguemnt description displayed in Simlytiks
mapsTo - Used by Items. argument used by Lucy in conjunction with “value”
simlytiks_group: Used by Items. Helps Simlytiks identify this field is from Lucy
source - Used by Items. Specifies where “value” will come from (data or custom)
dataTypes - Used by Items. Datatype of “value”. This will likely always be one element. But is a list of one element
optional - Used by Items. Is this argument optional for the task it belongs to?

options - Used by Items. Limited options for “value”. Example:

[
   {"id": "gini", "label": "Gini Impurity"},
   {"id": "entropy", "label": "Entropy"}
]

value - Optional. Used by Items. This is the value that is populated by the user–will be populated when returning from Simlytiks. If “value” is specified for the field in the Config Files, this is taken as the default value. This is not necessary. However, if the field is not optional, “value” must be populated at some point (default in Config Files or populated from Simlytiks)