Project

General

Profile

Feature #1073

Add variables to qkd-pipeline XML scripts

Added by Oliver Maurhart about 2 years ago. Updated over 1 year ago.

Status:
New
Priority:
Normal
Target version:
Start date:
23.06.2016
Due date:
% Done:

0%

Estimated time:

Description

qkd-pipeline should have some variables which can be used in composed configuration of qkd-pipeline XML-files.

These variables are:

p ... current pipeline id/name
m ... current module id/name
k ... current module instance number
n ... current module instance count
r ... current module role ('alice' or 'bob')

Example:

looking at the configuration like this:

<pipeline name="default" autoconnect="true" pipein="ipc:///tmp/qkd/default.alice.in" pipeout="">

    <module path="/usr/bin/qkd-cat">
        <config path="/etc/qkd/pipeline.conf" />
        <role value="alice" />
        <args value="--debug" />
        <log path="qkd-cat.alice.log" />
    </module>

    <module path="/usr/bin/qkd-sifting-bb84">
        <config path="/etc/qkd/pipeline.conf" />
        <role value="alice" />
        <args value="--debug" />
        <log path="qkd-sifting-bb84.alice.log" />
    </module>
</pipeline>

We have p = 'default', m = 'qkd-cat' or 'qkd-siftint-bb84' for the second module, k = 1 or each module and n = 1 for each module and r = 'alice'.

One now can create a XML-pipeline config like this:

    <module path="/usr/bin/qkd-sifting-bb84">
        <config path="/etc/qkd/pipeline-${m}-${k}.conf" />
        <role value="alice" />
        <args value="--debug" />
        <log path="${m}-${k}.${r}.log" />
    </module>

Which will hand out /etc/qkd/pipeline-qkd-sifting-bb84-1.conf as configuration file and will create a qkd-sifting-bb84-1.alice.log as log file.

This could ease up config file creation and reduce errors in these files.


Related issues

Related to Feature #1074: qkd-pipeline should support multiple parallel modules in a planNew2016-06-23

History

#1 Updated by Oliver Maurhart about 2 years ago

  • Related to Feature #1074: qkd-pipeline should support multiple parallel modules in a plan added

#2 Updated by Manuel Warum about 2 years ago

Sounds good, although I would restrict these placeholders to be only acceptable in path (and possibly URI?) attributes.

Might also declare a few more placeholders, such as local time (which might be handy for log files), machine's hostname, etc., depending on what is readily available.

#3 Updated by Oliver Maurhart about 2 years ago

Agreed:

h ... current hostname
t ... current UNIX epoch timestamp

#4 Updated by Oliver Maurhart almost 2 years ago

  • Target version set to v9.9999.8

#5 Updated by Oliver Maurhart over 1 year ago

  • Assignee set to Manuel Warum

Yet we need also some way to define an iterator. Say i_1 = 5000 + ${k}. This kann then be used for the different port numbers of each module.

How to define such?

Like this (?):

<module path="qkd-ldpc" instances=8>
    <iterator id="port" base="5000"/>
    ...
    <args value="--url-listen">tcp://127.0.0.1:${port}</args>
</module>

Is this somehow ok? What's the best practice for this when using XML files?

@Manuel: please comment on this.

#6 Updated by Manuel Warum over 1 year ago

  • Assignee changed from Manuel Warum to Oliver Maurhart

Hm. I had a little more time to think about it. Since this is XML, we might just as well drop the Bash-esque syntax altogether and do things like...

<module path="qkd-ldpc" instances="8">
    ...
    <args value="--url-listen">tcp://127.0.0.1:<port start="5000" /></args>
</module>

This would look better with syntax highlighting, probably.

The <port> tag in combination with the instances attribute of module might be enough. In any case, I would try to not use ${placeholders} unless really warranted. We can probably find some ways to go wild here, like...

  • <port start="8000" /> (start at port 8000 and count upward)
  • <port random /> (use any random available port)
  • <port use="8002,8004,8008" /> (use ports 8002, 8004, and 8008 respectively)

#7 Updated by Oliver Maurhart over 1 year ago

  • Assignee changed from Oliver Maurhart to Manuel Warum

Mhm, ok.

But this here limits us to the concrete example of "port". The bash-style variable technique is context agnostic and thus offers much more possibilities and flexibility. See the above example of the log directive:

<log path="${m}-${k}.${r}.log" />

We can go pretty wild with this... at some cost.

Is there any (successful or bad) example of utilizing such variables in XML files?

I do agree, your suggestion looks best with syntax highlighting but limits usage to "port". But then again you can add sophistic attributes...

Mhm, tough design decision. You do vote for dropping the variables and introduce dedicated new elements instead?
(I assume we can rewrite the log element with such XML tags as well, ...)

#8 Updated by Oliver Maurhart over 1 year ago

One additional thought: your <port> example is not necessarily bound to "port". We can define any number range with this, don't we?

So it might be rather <number start=8000/> or <number use="8002, 8004, 8010"/>...

#9 Updated by Manuel Warum over 1 year ago

  • Assignee changed from Manuel Warum to Oliver Maurhart

I'll sleep on it and try to find problematic edge cases for either approach tomorrow during my train ride. Maybe some sort of epiphany will come to me eventually.

Either way, we don't necessarily have to limit ourselves to <port>. It could be just one of several pre-defined variables, like <module_id>, <pipeline_id>, etc. Or even just a generic <var name="module_id"> instead of ${module_id} just to avoid using one language within another.

#10 Updated by Manuel Warum over 1 year ago

As per mini jour fixe from January 12th, XSLT might be a decent technology to make it as easy as possible to transform this new proposed approach into the old format.

#11 Updated by Benjamin Rainer over 1 year ago

Phew, XSLT may be a little bit bloated for such config files. I would suggest handling this directly in the source code. If you make changes to the (old) XML format, like adding tags/parameters or some other special stuff you will have to incorporate this into the XSL transformation code and the C code.

Also available in: Atom PDF