More than just an IDE

Ins and outs of Tcl/Tk

Tcl is a fairly easy-to-learn script language, but there are a few peculiarities that might be confusing for beginners as well as for advanced users. This page is meant to clarify its behavior; it is not meant as an introduction. We assume that you already have some basic knowledge about Tcl.
Hopefully, after having read this page, your will have a better understanding about Tcl and its behavior and will make less errors in the future.
 
  1. Tcl is a command language
  2. Why are control structures interpeted as commands?
  3. eval and {*}
  4. Can I use arbitrary variable names?
  5. The differences between "", {} and ()
  6. Using a " \" at the end of a line
  7. Control statements are parsed as Tcl commands
  8. How to treat comment lines
  9. Lists
  10. Namespaces
  11. Stack levels
  12. Tcl/Tk and IPEnv


1. Tcl is a command language

This means that the first token is always interpreted as a Tcl command or procedure. As a consequence, Tcl cannot cope with assignments such as a = b + c. The solution was to define define additional commands such as set and expr:

    set a [expr $b+$c]

Another consequence is, that — if applying the rules as strictly as Tcl does — control structures (if, switch, etc.), procedure definitions (proc) and comments (#) are interpreted as commands. This interpretation has a few other implications. One is that a comment block can cover several lines, without the need of commenting out each line separately. For example:

    # this a some comment \
      this is some additional comment

is the same as:

    # this a some comment
    # this is some additional comment

Another implication it that placing comment blocks is more restrictive than in C:

    string first "a" $str 	# find "a" in string str
    ==> wrong # args

See Section 7 and Section 8 for more details.

2. Why are control structures interpeted as commands?

This is related to how Tcl scripts are evaluated:
Step 1: The parser starts by checking the script for unmatched quotes and splits the script in groups of characters. These groups can be a single word or a group of characters without a single space, a variable (a $ followed by a simple word), a text string (quotes: ""), a fixed string (curly braces: {}), or a command (square brackets: []).

Step 2: The commands are identified and demarcated. The beginning of a command starts on a new line, or after a semi-column that was used to mark the end of a command.

Step 3: Before a command is executed all the arguments are pre-processed: The variables, backslashes, commands and {*}-constructions are substituted.
Fixed strings are left unchanged, including the backslashes (except at the end of a line). Curly braces in a text string are just ordinary characters withoud a special meaning and they do not shield off variables, i.e. dollar signs. An unmatched brace is no problem, but an open square brackets in a text string is always interpeted as the beginning of a command, even if if is unmatched.

    set x {{a $b \n} c}
    ==> {a $b \n} c
    set x "{a $b} c"
    ==> can't read "b": no such variable
    set x "a{b"
    ==> a{b
    set x "a[b"
    ==> (waiting for a close bracket: "[" is read as the beginning of a command)

Step 4: The commands are executed.
This cycle is repeated recursively to resolve and execute nested commands.

Example
Suppose that x equals {1 2 3 4}.
puts [lrange $x 1 end] is parsed as:

    "puts [lrange $x 1 end]"      # string
    puts [lrange $x 1 end]        # separate arguments
    puts [lrange {1 2 3 4} 1 end] # substitution
    puts {2 3 4}                  # execution lrange
    ==> 2 3 4                     # execution puts

Steps 1-3 are executed without any knowledge of the commands. This mechanism has the advantage that the command name can also be a variable, or a user-defined procedure that is not yet available during compilation. Suppose, for example, that we have a variable that has value "#" and is used as a command name, then the command line will be interpreted as comment. If there is a backslash at the end of the line, this line and the next one are concatenated, independent of the command. So inevitably, both lines are always evaluated as a single command:

    set cmd "list"
    $cmd arguments of $cmd\
        continued on next line
    ==> arguments of list continued on next line

but if the command name changes, it will be executes in the same way:

    set cmd "#"
    $cmd arguments of $cmd\
        continued on next line
    ==> (empty line)

It is not recommended to apply these constructions arbitrarily, but they are essential to Tk and other Tcl extentions where the widgets names that are dynamically create are stored in Tcl variables.

Remark
System error messages might seem the same, but they can be generated on a different level. In the following example, a fixed string is passed to subst. subst analyzes the string and throws an error (step 4):

    subst {$a b}
    ==> can't read "a": no such variable

If the same input would be passed as a text string, the error message is caused by the parser at step 3, but with the same results:

    subst "$a b"
    ==> can't read "a": no such variable

3. eval and {*}

If you want to execute a command that is stored in a string, you can use eval. If you have, for example, an interface that reads the command arguments from an entry widget and the options from a combobox, the command can be executed as:

    eval cmd -opt $value $arguments

eval interprets each index of each argument as a separate argument of cmd. If value is a text string, this may cause problems, because you would want to read it as a single argument. Putting $value in quotes doesn't make any difference, because This can be inhibited by adding the list command: [list $value], or: "eval [list cmd -opt $value $arguments]". This will replace $value by a list containing a single element. In Tcl version 8.5, a new syntax element was introduced solving this annoyance. Instead of using the eval command, you can now write {*} in front of the variables that are to be interpreted as separate arguments. The command above can be written as:

    cmd {*}$arg1 $arg2 {*}$arg3

{*} can be used more than once in the same command:
Another example that you can test easily is:

    set str "strerr test"
    puts $str
    ==> stderr test
    eval puts $str
    ==> test

or use {*}:

    puts {*}$str
    ==> test

eval still remains useful, if, for example, you want to execute a command line that is read from a console as a whole.

4. Can I use arbitrary variable names?

• A variable name can be an arbitrary string of characters. Names such as "5", "a+b" or even "\$x; {2" are completely legal, but not recommended. Be aware that a variable name can also be an empty string. If you are a beginnner, this might be confusing, because you can get cryptic error messages. Suppose that y already exists as an empty string and by mistake you have written "set $y" instead of "set y". You will get the error message:
can't read "": no such variable.
The syntax was alright, but you will get a runtime error.

• You can concatenate variables by writing them together, without white spaces in between: $str1$str2.
If you want to append a string to a variable you can use curly braces to demarcate the variable. set y ${var}str is the same as writing:

    set y $var
    append y "str"

or:

    append y $var "str"

y is empty or has not yet been defined. Using append is alway the safest approach. If the string contains white spaces, and you would want to use quotes, you will notice that a quotation mark is interpreted as an ordinary character and is appended to the variable.

    set var 0
    set y $var" 1 2 3"
    ==> wrong # args: should be "set varName ?newValue?"
    set y $var"\ 1\ 2\ 3"
    ==> 0 "1 2 3"
    set y $var\ 1\ 2\ 3
    ==> 0 1 2 3
    set y $var"
    ==> 0"

If an expression is not ambiguous, curly braces are not needed, for example $a.b simply appends .b to $a. All operators and punctuation character, except for "::" are interpreted as ordinary delimiters for variables. You will get an error message if you write $z::, because z will be interpreted as a namespace. A single column, however, is also read as a delimiter. For example: $filename: does not exist.

5. The differences between "", {} and ()

() are only used to indicate an array index.
Both "" and {} mark the beginning and the end of a string. The difference is that curly brackets inhibits the substitution of variables, commands, and backslashes. This does not mean, however, that a fixed string is always passed as is. The exception is, when a backslash is used to concatenate lines. In this case, the backslash and the leading white spaces on the next line are replaced by a single space:

    set v {a\
	   b}
    ==> a b

If v is to be interpreted as a list, the backslash is redundant. Note that a backslash substitution is defined the same for text strings:

    set v "a\
	   b"
    ==> a b

If you do not use a backslash, all character between the quotes or braces — including the newlines — are retained:

    string length {a\
	   b}
    ==> 3
    string length {a
	   b}
    ==> 10 (this includes the newline)

Curly braces cannot be used to define a string containing a single brace:

    set x {\{}
    ==> \{

If you do not insert a backslash, the character is read as an unmatched brace, but if you do so the backslash is retained and you cannot get rid of it. This contradiction can only be solved afterwards, by using the subst command:

   set x {a\{b}
   => a\{b
   subst $x
   ==> a{b

If you want to retain commands or variables in a fixed string, you must add -nocommands and/or -novariable to the subst command to inhibit complete substitution:

    set x {a\{b $c}
    ==> a\{b $c
    subst $x
    ==> can't read "c": no such variable

Insert -novariable to inhibit $c from being replaced:

    subst -novariable $x
    ==> a{b $c

Note that if a string contains an unmatched square bracket that is not preceded by a backslash, you will also need to add the -nocommand option, even if you do not intend to use the bracket a the beginning of a command:

    set x {a[b}
    ==> a[b
    subst $x
    ==> missing close-bracket
    subst -nocommand $x
    ==> a[b

6. Using a " \" at the end of a line

There are three case where backslashes to escape newlines is meaningful.

• The first one is already mentioned above. It can be used to remove redundant leading spaces in a string covering more than a single line. Note that the white spaces in front of the backslash are not affected. A backslash does not require a delimiter:

    set x {a \
	   b}
    ==> a  b (two spaces in between. not just one)

If you want to get rid of the leading spaces only, and still want to retain the newlines, you can write:

    set txt "\
        line1\n\
        line2\n\
        line3"

• The second case is if you want to continue a command on the next line. Otherwise arg3 will be interpreted as a new command name:

    command arg1 arg2 \
        arg3 arg4 ...

This construction can be hidden in an embedded command:

    array set test [list \
        index1 val1 \
        index2 $val2 \
        index3 "$val3 $val4"
    ]

The backslashes are needed, not to group the text between the square brackets — brackets are always automatically matched — but to indicate that the list command does not stop at the end of the line . Consequently, the last backslash (before the close bracket) can be left out, although it wouldn't do any harm.
We could also use quotes and leave out the backslashes at all:

     array set test "
        index1 val1
        index2 $val2
        index3 [list $val3 $val4]
    "

• The third case is a trick to let a Tcl script behave differently under bash and in Tcl:

    #!/bin/sh
    # The following line is hidden in Tcl, but not in bash \
    exec wish $0 ${1+"$@"}
    # Tcl commands follow

If you source this script in wish, the exec command is hidden and the Tcl commands are evaluated. If you run this script in bash, then exec is executed which launches the current script in wish. wish will be running in the foreground. If you exit wish, this script will also end and the Tcl commands will not be evaluated by sh.

In all other cases, a continuation token is redundant or might even be wrong.
You may read constructions such as:

    package ifneeded tdbc 1.0.6 \
        "package require TclOO 0.6-;\
        [list load [file join $dir tdbc106t.dll]]\;\
        [list source [file join $dir tdbc.tcl]]"

The first backslash is needed to link the last argument to the package command. The other backslashes are redundant, because the lines are already joined together. Having placed backslashes, however, the newlines are removed as the natural command delimiter, and you are forced to insert the semi-columns. In this example, the list commands are needed to keep the file names together as a single string (see: the eval command). A better syntax would be:

    package ifneeded tdbc 1.0.6 [subst {
        package require TclOO 0.6-
        load "[file join $dir tdbc106t.dll]"
        source "[file join $dir tdbc.tcl]"
    } ]

Note the additional whitespace between } and ] on the last line.

In the next example, the backslashes are definitely wrong. Purpose was to create a button that changes the background color of another widget and does some other stuff. w is the local name of a widget already created that you want to use in a binding:

    # WARNING: the backslash is redundant and evokes an error
    set b [button .b]
    bind $b <Button-1> " \
	$w configure -bg red \
	# additional Tcl commands
    "

Removing the backslash is an option, but notice that w is already replaced during compilation and does not exist as a variable when the binding is executed. A general solution would be to define w as a global variable, or a namespace variable. If you did so you could write:

    bind $b <Button-1> {
	$::w configure -bg red
	# additional Tcl commands
    }

7. Control statements are parsed as Tcl commands

The fact the control statements are parsed as commands, has a few implications.

• The conditions are placed in curly braces instead of parentheses.

• Sometimes you can see:

    proc fnc x {...}; # no curly braces around the argument

instead of:

    proc fnc {x} {...}

This is correct, but it is a sloppy style of programming and is not recommended because it makes the code less readable. Moreover, leaving out the curly braces may not always be correct. The following code is executed correctly:

    if $x {
	# do something
    }

But for different reasons: $x is replaced by its value before the if command is evaluated (step 3). If you place curly braces around $x, then $x is evaluated by the if command itself (step 4). As a user you will not notice the difference. If we would execute the following code, however, leaving out the curly braces, we would end up in an infinite loop, printing "9" indefinitely:

    set i 10
    while [incr i -1] {
	puts $i
    }

The switch statement is programmed differently, and does NOT evaluate the flag. So it must be replaced in advance before it is executed, and curly braces are definitely wrong:

    switch {$x} {	# compares the string "$x" and not its value
	# switches
    }

• Why can't we just write:

    if {$a == str} {...}

instead of:

    if {$a == "str"} {...}

It is because the condition is passed as a fixed string to the if command and is evaluated as an expression. A variable can be interpreted both as a number and as a string, and does not need to be placed in quotes, but the syntax "expr $a == str" is incorrect.
This means that the condition does not require a strict format containing mathematical operators, and can contain any expression or procdedure that returns an integer: For example:

    # x contains an integer
    if {$x-5} {
	puts {$x does not equal 5}
    } else {
	puts {$x equals 5}
    }

Similarly, we can write:

    for {set i 10; set j 1} {$j} {incr i -1; set j $i} {
	puts $i
    }

instead of:

    set j 1
    for {set i 10} {$j != 0} {incr i -1} {
	puts $i
	set j $i
    }

8. How to treat comment lines

• Positioning comment lines is more restrictive than in C. It might help to realize that comments are just special Tcl commands. You wouldn't write:

    string first "a" $str 	set str "test"

then why writing:

    string first "a" $str 	# set str "test"
    ==> wrong # args: should be "string first needleString haystackString ?startIndex?"

The comment is parsed as arguments of string. You can solve this problem by inserting a semi-column between the two commands.

Generally, you can always write a comment on a new line. This can be anywhere in a script, but not inside an argument, except in the body of a statement. This also includes the switch statement:

    switch $n { # comment here is ok
      # comment here is also ok
      1 {... commands ...}
      # and here
      2 {... commands ...}
    }

• The parser checks whether the command is complete and this includes a comment block. As a result, commenting out a control statement as follows goes wrong:

    # while {$x} {
	(do something)
    # }
    ==> unmatched open brace in list

You can add a backslash in front of the unmatched open braces in the comment lines. An unmatched close brace does not cause any problem.

9. Lists

A list is not an intrinsic property of a string; it is just another interpretation. This interpretation is made by the command, and not by the parser. For example, lindex interprets the first argument as a list, whereas split interprets the same string as a fixed text:

    lindex {{$a b} {c d}} 0;	# interprets argument 1 as a list
    ==> a $b

but:

    split {{$a b} {c d}};	# interprets the argument as a string
    ==> \{\$a b\} \{c d\};	# list of four elements

• A list can be created by parts as an ordinary string. The interpretation of the string comes afterwards:

    set lst {"a $b"}
    append lst " \{"
    append lst "c \$d\}"
    puts $lst
    ==> "a $b" {c $d}
    # we can read lst as an ordinary list:
    lindex $lst 0
    ==> a $b
    lindex $lst 1
    ==> c $d

• The lists,

    {"a $b" {c $d}}

and

    {{a $b}    {c $d} }

are identical. The quotes and braces are not a part of the elements and are merely used to mark the beginning and the end of the elements.

• A third way to group the elements of a list is by using backslashes in front of the white spaces in the elements: {a\ $b c\ $d} represents the same list above. Backslashes are automatically inserted if the string contains braces. For example:

    list "a { b"
    ==> a\ \{\ b

this, in contrast to:

    list "a b"
    ==> {a b} (this could also be written as a\ b)

The backslashes can be seen as a necessary internal representation. If you stick to the list-commands, you might not even notice them at all:

    lindex [list "a { b" "c d" ] 0;	# same list as above
    ==> a { b

• Not all strings can be interpreted as lists. A list requires that the string does not contain unmatched braces or quotes. If you want to get the procedure names in a Tcl script by checking it line by line, then the following piece of code will fail:

    if {[regexp {\s*proc } $line]} {
        # the next command will fail because $line is not complete:
        lappend procs [lindex $line 1]
    }

You can check completeness with "info complete $string".

• Tcl contains a set of commands, specially designed to process lists. They are:

    lappend lassign lindex linsert llength lmap lrange lreplace lreverse lsearch lset lsort

The first argument of these commands always refers to a list.
lappend and lset read the list by name, the other commands read the list as a variable or a literal list.

lappend and lset return the modified list, both as an argument and as the return value:

    set lst2 [lappend lst1 elem]
    # lst1 and lst2 are identical. lst1 is also modified

The list commands that modify the list, reformat the input string according to a uniform format using curly braces and all white spaces in between the elements are replaced by a single space:

    set lst {"a $b"    {c $d}  }
    lappend lst "e f"
    ==> {a $b} {c $d} {e f}

• There are four Tcl commands to create new lists. They are: concat, lrepeat, split, and list.

10. Namespaces

A namespace is created as namespace eval name script. The script can be used to create namespace variables and namespace procedures, or execute commands, or can be empty.
• Once a namespace is created, you can dynamically add or delete as many procedures and variables as needed. You can use namespace eval once again, or you can use set and proc, writing the namespace name expicitly. The following statements are identical:

    namespace eval ns {
        # the commands are executed inside a namespace
        variable x 10;    # creates ns::x
        set y 0;          # creates ns::y
        proc fnc {} {     # creates ns::fnc
            puts "creating [namespace current]::fnc2"
            proc fnc2 {} {
                puts "current frame: [info frame [info frame]]"
                namespace eval {} {
                    puts "creating [namespace current]::fnc2"
                    proc fnc2 {} {
                        puts "current frame: [info frame [info frame]]"
                    }
                }
            }
        }
        fnc;              # executes ns::fnc. creates ns::fnc2
        fnc2;             # executes ns2::fnc2. creates ::fnc2
        ::fnc2;           # executes ::fnc2
        namespace eval ns2 {};  # creates namespace ns::ns2
    }

and:

    # the commands are executed in the global namespace
    set ns::x 10
    set ns::y 0
    proc ns::fnc {} {
	proc fnc2 {} {puts [info frame [info frame]]}
	namespace eval {} {
	    proc fnc2 {} {puts [info frame [info frame]]}
	}
    }
    ns::fnc
    ns::fnc2
    fnc2
    namespace eval ns::ns2 {}

• The procedures and variables that are created by set, array, source, and inside proc and "namespace eval" are added to the current namespace. If this is not what you meant, then it can be overruled by calling "namespace eval {} {... commands ...}" from within another namespace.

  The variable command declares a variable in a namespace, but if you do not assign a value to the variable, it cannot be accessed and unset:

    namespace eval ns {variable var}
    info vars ns::*
    ==> ::ns::var (i.e. ::ns::var exists)
    set ns::var
    ==> can't read "ns::var": no such variable
    unset ns::var
    ==> can't unset "ns::var": no such variable

• Global variables and procedures are stored in the global namespace. Its name is the empty string. Write ::varname to access a global variable from within another namespace or procedure, or to declare it as a global variable. You can also use the global statement and leave out ::.

• An callback event is always executed in the global namespace, so if you create a variable at this level, it will be retained as a global variable. For example:

    bind $w <Button-1> {
        set pos [%W index @%x,%y]
        # etc.
    }

pos can be accessed as a global variable from anywhere.

11. Stack levels

If you enter a procedure you move to a deeper stack level. Inside namespace eval you will also enter the next stack level. The top level is the global namespace, and is level 0.
The Tcl command upvar and uplevel give full access to another stack level. If you need to know, for example, the namespace from which a procedure is called, you can write: "set ns [uplevel ::namespace current]"
uplevel concatenate the arguments and executes the string. The default uplevel is 1. You can often see constructions such as: "set ns [uplevel 1 [list ::namespace current]]". Here, both 1 and list are redundant. For the sake of clear coding, the best solution is to apply the same rules as for eval and use list only for arguments containing a list that must be retained. Instead of:

    uplevel 1 [list catch [lreplace $args 0 0 [lindex $cmds $n]] ::result ::opt]

write:

    uplevel catch [list [lreplace $args 0 0 [lindex $cmds $n]]] ::result ::opt

The most common implementation of upvar is to pass procedure arguments by pointer. Actually, there are no pointers in Tcl, but they are simulated as:

    proc copy {arg1 arg2} {
        upvar $arg2 lvar2
        set lvar2 $arg1
    }
    set var1 10
    copy $var1 var2
    set var2
    ==> 10

upvar links a local variable lvar2 with a variable in the stacklevel from which copy is called. It does not create a new variable.
$arg2 does not need to exist before copy is called. Setting lvars will change or create $arg2.
You could also write, seemingly with the same results:

    proc copy {arg1} {
        upvar var2 lvar2
        set lvar2 $arg1
    }
    set var1 10 
    copy $var1
    set var2
    ==> 10

But then, it will always create a variable with the same name.

12. Tcl/Tk and IPEnv

IPEnv is a multilingual IDE, but it has strong roots in Tcl/Tk. If you are programming in Tcl/Tk, you can profit from all the advantages. that IPEnv has to offer: sophisticated automatic code completion, clever goto and find functions and it recognizes Tcl procedure prototypes. You can load your own plug-ins written in Tcl/Tk.
IPEnv has a built-in multi-threaded console in which you can execute Tcl/Tk commands, interrupt Tcl commands and system commands as well as, run Tcl scripts in the background and much more. Just give it a try and you will realize what you are missing.
IPEnv comes with the latest stable version of Tcl/Tk, and several additional Tcl extensions, including BLT and TclProDebugger.