qooxdoo includes its own build system which is preconfigured through the Makefiles included in the applications and the skeleton. However it is possible to use the more hidden features directly in an own Makefile (, ant, cmake, ...) solution

The build itself makes it possible to compress, optimize, deploy or even document the code as well as copies all the required resources for all qooxdoo like JavaScript projects.

It is possible to build your own qooxdoo applications (including other qooxdoo like libraries). You can configure it to include your custom classes and resources to generate an optimized application that is “linked” against the qooxdoo framework and that can be deployed anywhere in your network.

• Multiple class paths: e.g. you are able to include the qooxdoo source code and your own classes to build the final application. (Multiple namespaces are possible and suggested, too)
• Automatic dependency resolver: Scans all classes for cross-dependencies and sorts files according to the results
• Advanced include/exclude system with support for single classes (including wildcard support) and modules. You can also disable the dependencies for include and/or exclude (to make it possible to include/exclude a single class without their dependencies).
• Module system: Group classes from multiple packages to virtual groups and use them for your include/exclude logic.
• Namespace support: You can structure your own classes into namespaces. The API browser allows you to navigate through the namespace hierarchy. Namespaces are also supported for the include/exclude feature (with wildcards etc.).

• Compression: The result scripts are compressed. The current implementation automatically removes comments, new lines, indents and other additional whitespaces from your files. It keeps advanced code fragments like regular expressions, strings, ... You just to make sure to add semicolons after each command. Additional semicolons, which are needed for compression will be added automatically. It is possible to add new lines after each semicolon to the result script file. It is also possible to add markers to this file, to the starting point of new files. This allows easier debugging if there are missing semicolons etc. in your source code.
• API Documentation: You are able to build an API viewer including your own classes. You can choose to only include the classes needed by your application or all available classes.
• Source generator: The build system is also able to generate a “source version” of the final JavaScript. This script contains information which will be dynamically written to the target document. This way it is possible to develop an application using the source classes, without to do a rebuild on every little change. (You only need to reexecute this if you move or rename script input files for example.). This version is really helpful while developing applications, because syntax and other problems reported by a debugger will directly show you the problematic part in your code.
• Resource synchronisation: It is possible to automatically syncronize all needed resources (images, additional CSS, HTML, ...) of your application to your destination directories. This allow it to build a self-contained distribution of your application. The resource system is class based (e.g. a class needs resource A and C) and this way works together with the include/exclude feature mentioned above.

• Variable shortening: Further compression is possible using the variable compression introduced in recent qooxdoo versions. If the corresponding option is enabled all local variables and parameters get shortened to something like ‘$4’. This option is quite stable and is enabled by default in all applications delivered together with qooxdoo.
• String optimization: Recent Internet Explorer versions have problems regarding the performance of typical string operations e.g. comparing or concatting them. One bottleneck, is if a string just exist temporary in a comparison or concationation. In this case Internet Explorer seems to create a new string instance for each iteration (which is relatively slow) (Especially with the bad scaling behavior in Internet Explorer ⇐ 6). With this new feature in our build system it is possible to automatically optimize strings for optimal performance. You just need to activate the corresponding option on the command line. This feature seems to be safe enough for general usage. It is already enabled in all applications delivered with the qooxdoo downloads.
• Runtime settings support: With the build system you can also define the runtime settings of your application. For example the following stuff is already possible:
  • Paths to images and other resources
  • Initial themes
  • Class settings (especially for singletons)
• Encoding support: The script files defined as input are automatically checked against your defined encoding (default is UTF-8). The output encoding for XML files and JavaScript files could be defined separately.

• Tokenizer and tree generator: The whole system is built up on our own tokenizer and tree generator (AST) (coded entirely in Python).
• Developer support: You are able to print out lists of all known files, all known modules, the sorted include list and the dependency information gathered from the JavaScript source code. It is also possible to print out the tokens or tree of the JavaScript source to separate files to detect errors of the tokenizer or tree generator on specific code.
• JSON and XML export support: You can export the code tree of your files (all included files) to both, XML and JSON.
• Intelligent cache: Many results of the system can be cached to separate files. This improves the performance of the build process on repeated execution of the process.
• Configuration files: You can define your complete settings in config files. This makes them easier editable. The parameters of the configuration file are identical to the ones of the command line but it adds some new stuff e.g. multiple output modules, comments between your arguments, easier syntax for complex arguments, ...
• Hint support: With the same system, with which you define the modules, you can define additional settings and other hints:
  • #use(classname): This class uses classname in the runtime. Normally not needed (autodetection)
  • #require(classname): This class uses classname in the loadtime and maybe the runtime, too. (You can upgrade automatically found items to load requirements using this.)
  • #after(classname): The actual class should be loaded afterclassname. classname will be added to the dependencies of this class.
  • #load(classname): The mentioned class will be automatically included if the actual class is selected for inclusion. classname will be added to the dependencies of this class.
  • #optional(classname): The defined classname is optional and is not needed, but used by this class. This is a special hint for the autodependency resolver to ignore these dependencies. This could be useful if the class use control flow statements to handle the missing classname. Using this makes it possible to build smaller js-packages in some cases.
  • #module(modulename): Add this class to the module modulename.
  • #id(identity): Define the internal ID of this class using identity. Normally this is not needed in a qooxdoo environment because all classes use the defineClass statement which is also supported.
  • #resource(namespace.resourceid:localpath): Defines that the stuff in localpath should be part of a resource named resourceid. You can have multiple #resource items with the same IDs to group them together under the same ID. The stuff defined by this definition will be included with this class. If the corresponding class is excluded the resources are excluded, too. You can more detailed filter which files should be included using the embed statement beyond. The resource must be defined in a class which is part of the same project. This basically means that the resource files should be, must need not to be, part of the directory structure of the library or application where they are included.
  • #embed(namespace.resourceid/directory/file): The embed hint allows you to explicitely define the files from each resource needed by a class. This allows completely optimized builds e.g. only use some icons of the whole icon themes included. You can even support shell like expressions e.g. resourceid/directory/*.gif. resourceid is the ID defined by a resource statement in any of the included classes. #embed and #resource need not to be defined in the same class. You can access any resourceid defined in any namespace in each of your classes.

We use $qx for your custom checkout/download directory of qooxdoo in the future. To use the build system just execute $qx/frontend/framework/tool/generator.py. All files and directories defined as parameters must be relative to the current working directory or absolute.

• -h, --help: Print help screen.
• -v, --verbose: Be verbose
• -q, --quiet: Be quiet (default)
• --from-file=FILENAME: Read configuration from FILENAME
• --export-to-file=FILENAME: Export command line configuration to FILENAME.

• --class-path=DIRECTORY: Define the directory as input for JavaScript class files. The content of the directory should immediately start with the first namespace folder. There shouldn’t be any JavaScript files directly in this folder. DIRECTORY is a directory reachable from the current machine. Remote access using http, sftp, etc. is not supported.
• --class-encoding=ENCODING: Define the encoding of the input script files. The files will be converted to python’s internal string representation. All files of the class-path must conform this encoding type. Otherwise you will get an exception from the build script. ENCODING is any possible encoding. See also at wikipedia.

To use the compression feature of the build script you must add the parameter --generate-compiled-script to select the job and then must add --compiled-script-file FILENAME to define the output file for the compiled script. FILENAME represents a full qualified file name.

$ cd $qx/frontend
$ framework/tool/generator.py --class-path framework/source/class \
  --generate-compiled-script --compiled-script-file custom.js

You are also able to enable the cache feature of the file loader. This dramatically increases the performance of the build system after the first execution.

$ framework/tool/generator.py --class-path framework/source/class \
  --generate-compiled-script --compiled-script-file custom.js \
  --cache-directory .cache

This stores all cache files to the .cache folder in your working directory. The cache folder stores information about all included files. You cannot separate the cache paths for different class paths.

You can configure the compression method a bit. You can add new lines after each semicolon (--add-new-lines) and the IDs of your classes (--add-file-ids). These options are helpful to find errors caused by compression. Mostly these are due to missing semicolons in the script source code.

You can also define multiple script inputs e.g. qooxdoo and your own project. With each script input you start a new set. A set normally should define all these arguments, depending on the actions choosen:

1. --class-path=DIRECTORY
2. --class-encoding=ENCODING
3. --class-uri=PATH
4. --resource-input=DIRECTORY
5. --resource-output=DIRECTORY

All parameters are generally optional. But keep in mind that if define a argument for one script input to define it for the others too. You can imagine each argument as an indexed array. Missing arguments will cause interesting errors ;)

You are able to define as many sets as you like. This allows you to build a distribution of a whole application from three different folders, where each has own resource files and copy all together inside a self-defined distribution folder with the structure you wish to have.

To keep it transparent it would probably the best to group these five arguments together for each script input e.g.:

$ framework/tool/generator.py \
  --generate-compiled-script --compiled-script-file custom.js \
  --class-path source-directory-1 --class-encoding utf-8 \
  --class-path source-directory-2 --class-encoding iso-8859-15

I’ve ignored the other arguments here as we explain them later. As long as you just use the compression feature, it’s ok to only define these two options.

To customize your build you can define which classes to include or exclude from your build.

To include a specific class including their dependencies add the following argument to your argument list --include classname. classname is the full qualified classname in this case (including the complete namespace). For example: qx.ui.toolbar.ToolBar.

You can also include modules. A module is a set of classes which are often used together e.g. qx.ui.tree.Tree and qx.manager.selection.TreeSelectionManager. Using modules you can keep your include lists shorter.

$ framework/tool/generator.py --class-path framework/source/class \
  --generate-compiled-script --compiled-script-file custom.js \
  --include tree --include qx.ui.embed.Flash

This will include the tree module and the class qx.ui.embed.Flash. Both including with their dependencies. You can also use a comma to separate multiple includes:

$ framework/tool/generator.py --class-path framework/source/class \
  --generate-compiled-script --compiled-script-file custom.js \
  --include tree,qx.ui.embed.Flash

To just include/exclude the defined modules and classes and to ignore the dependencies add the argument --include-without-dependencies or --exclude-without-dependencies. This is a global flag for all includes or exclude definitions. Generally you want to use this more often for exclude than for include.

The build script is able to copy resources (images, flash movies, HTML files, ...) of your classes to a build/dist folder, too. This allows you to automatically generate a self-contained build version of qooxdoo. You can enable the resource copy feature adding --copy-resources to your argument list. But wait. You are not already done. There is a lot of magic included, but we couldn’t save you from defining the input and output directory of the resources. For qooxdoo the input folder is simply framework/source/resource. For your own projects you must know this yourself. The second thing is the output directory.

$ framework/tool/generator.py --copy-resources \
  --class-path framework/source/class \
  --resource-input framework/source/resource --resource-output mybuild/resource

As explained above the parameters used here: --resource-input and –resource-output are members of a set. For each set you can define your own resource input and output folder. I will demonstrate this in a small example:

$ framework/tool/generator.py --copy-resources \
  --class-path framework/source/class \
  --resource-input framework/source/resource \
  --resource-output mybuild/resource/qooxdoo \
  --class-path ~/myproject/source/class \
  --resource-input ~/myproject/source/resource \
  --resource-output mybuild/resource/myproject

In this case you define two sets. Each set with it’s own resource folder for in- and output.

By the way: You can also enable both jobs together, compiling and resource copying. For example:

$ framework/tool/generator.py --copy-resources --generate-compiled-script \
  --compiled-script-file mybuild/script/build.js \
  --class-path framework/source/class \
  --resource-input framework/source/resource \
  --resource-output mybuild/resource/qooxdoo \
  --class-path ~/myproject/source/class \
  --resource-input ~/myproject/source/resource \
  --resource-output mybuild/resource/myproject

You are also able to exclude classes from the resource copy. The system will only copy resources from the classes you have included. (If no include is defined, all resources of all classes are copied.)

The source version is a topic itself and sometimes complicated to explain. Especially for typical users of other rich programming languages. JavaScript is able to run without being compiled so there is normally no reason to put in a compiler step into your application. The major problem without compiling (concatting) is that you must include all your classes manually inside your HTML file(s). That’s not an easy job. Especially because the most time the classes depends on each other (inheritance) and need a special order to be included/loaded.

Working with qooxdoo, you maybe already know that there are 200+ classes available. To include all of them in your HTML file(s) is just impossible. Before our current solution was available the people developing applications using qooxdoo often have used the built qooxdoo script and included their own classes manually into the HTML files. This results into the possibility to be able to easily modify your own classes without any additional compiler step between each change. That is what the most people need. However, using this solution, it’s not that easy to find out a misbehavior or even bug in qooxdoo. The debug informations you get from a compiled script, where all code is in one line is not really helpful.

We kept this possibility. But there are new possibilities available, which dramatically simplifies your work. With the same advanced logic of the compiler, you can generate a JavaScript file which will dynamically include the original class files from the original directories. This means you can use the source version of both, the 200+ qooxdoo classes and the, maybe, 20+ own classes. The build system will automatically sort your files according their requirements and generates a JavaScript file which could be directly included in your application. Using this method it’s also possible to easily toggle between source and build using some session variable of your webapplication for example.

To make it possible that this generated JavaScript can find the source files of your JavaScript classes you must define the path, where the classes are available from your filesystem or web server. In case of a web server you must be sure the class files are accessible for the visitor first. You can do this adding --class-uri for each set to your argument list.

$ framework/tool/generator.py --class-path framework/source/class \
  --generate-source-script --source-script-file custom.js \
  --class-uri /data/script/qooxdoo/class

This is comparable to the example with the compiled script as output. We have just replaced two parameters with their source neighbour and added --class-uri for the newly required information where the source script file can find the other classes.

To customize settings like image paths, default themes, etc. you can generate runtime settings with the build system. Each default setting can be overwritten. Settings are available class-wide.

Common settings and their default value:

• qx.core.Object
  • enableDisposerDebug: false
• qx.io.Json
  • encodeUndefined: true
  • enableDebug : false
• qx.io.image.ImagePreloaderSystem
  • timeout: 3000
• qx.io.remote.RemoteExchange
  • enableDebug: false
• qx.lang.Generics
  • enable: false
• qx.lang.Prototypes
  • enable: false
• qx.theme
  • Theme: “qx.theme.ClassicRoyale”
• qx.colorTheme
  • Theme: “null”
• qx.borderTheme
  • Theme: “null”
• qx.fontTheme
  • Theme: “null”
• qx.widgetTheme
  • Theme: “null”
• qx.appearanceTheme
  • Theme: “null”
• qx.iconTheme
  • Theme: “null”

You can overwrite each setting separately. Just add the following argument: --define-runtime-setting key:value.

$ framework/tool/generator.py --class-path framework/source/class \
  --generate-compiled-script --compiled-script-file custom.js \
  --define-runtime-setting \
  qx.manager.object.ImageManager.iconTheme:qx.theme.icon.nuvola

For each overriden setting, you must include a “–define-runtime-setting”. As you can see the key contains two elements, the classname and the name of the setting, separated by a dot.

The qooxdoo distribution comes with a full-blown API viewer. The only thing you must fill yourself is the json data needed to fill the application with content. Inside the qooxdoo distribution the API viewer resides into frontend/apiviewer. There is also a Makefile available to create the build version of the API viewer. The build version is self-contained. You should be able to copy it to any other directory on your system or any public server.

It is also possible to include custom classes into the tree of the API viewer. The API viewer uses a JavaScript file with a large JSON data structure. You can recreate this file, adding your own directories and classes. Then you can simply replace script/apidata.js inside your built.

$ framework/tool/generator.py --generate-api-documentation \
  --class-path framework/source/class \
  --class-path ~/myproject/source/class \
  --api-documentation-json-file api/build/script/apidata.js

This will overwrite the data generated through the Makefile of qooxdoo. Maybe you want to generate the json file to some different destination and also want to generate the API viewer there.

It is also possible to use the API viewer folder as a library like folder and to create the API viewer in your own directory structure - automatically. This is used by the skeleton delivered with qooxdoo. The idea basically is to define the API viewer classes as an additional class path. You must also define the resource folder inside the API viewer as an resource input. Then is is possible to only include the module apiviewer to build an automatically optimized apiviewer for your project. This step does generally does not create the data file apidata.js. This file must created in a second step (separate execution of generator.py). This reason for this is that the includes for the API viewer built and the data file are normally different.

There are other target to optimize your application classes. For example you can automatically pretty print them.

$ framework/tool/generator.py --pretty-print \
  --class-path framework/source/class \
  --class-path ~/myproject/source/class \
  --include-without-dependencies myproject.*

There are a couple of additional options that affect the pretty printing, like the choice of the indent string and handling of opening curly braces. Invoke the generator with framework/tool/generator.py –help and scan for “pretty”.

A less obstrusive alternative, which just corrects some indent issues and codepage things is the --fix-source option.

$ framework/tool/generator.py --fix-source \
  --class-path framework/source/class \
  --class-path ~/myproject/source/class \
  --include-without-dependencies myproject.*

We have also made it possible to combine all of the arguments explained above. You can combine the copying of resources with the generation of the compiled version (build). You can create the the source and build version together. Just try yourself.

As you have seen, especially when combining multiple options, the argument lists gets longer and longer. You have the possibility to put these informations inside a shell script, Makefile or something similar. Another possibility is to use the configuration file feature which is included in the build system.

$ framework/tool/generator.py --from-file ~/myproject/myconfig.cfg

The config file is a simple text file. The parameters inside the config files are the same then on the command line, but it’s only possible to use the long options e.g. you must use --include instead of -i. Inside the config file you must remove the dashes on the left of the parameter name. The divider between key and value is a = symbol.

generate-compiled-script 
class-path = framework/source/class
compiled-script-file = custom.js

Of course, you can also place comments, spaces and new lines inside the file:

generate-compiled-script
compiled-script-file = custom.js

################
# set 1
################
class-path = source-directory-1
class-encoding = utf-8

################
# set 2
################
class-path = source-directory-2
class-encoding = iso-8859-5

You can find sample configuration files under framework/tool/presets:

$ framework/tool/generator.py --from-file framework/tool/presets/io.dat

The mentioned io.dat is a definition to build a simple package which contains qooxdoo and all classes needed for remote communication.

You can also convert your existing command line to a config file using this command:

$ framework/tool/generator.py --export-to-file myconfig.cfg --option 1 --option 2 [...]

The parameters to define the runtime settings, as explained above, is somewhat complex. Inside the config file this is defineable in two ways:

define-runtime-setting = qx.manager.object.ImageManager.iconTheme:qx.theme.icon.nuvola

or:

define-runtime-setting[qx.manager.object.ImageManager.iconTheme] = qx.theme.icon.nuvola

The second separates the key from the value in a better way. The most people find this syntax easier.

The generator.py has some other options, too. For example to create debug output, to write the syntax tree and tokenizer output to files, etc. You can get an overview over these options using:

$ framework/tool/generator.py --help