From Van Essen Lab

Revision as of 18:50, 7 December 2011 by Tim (Talk | contribs)
Jump to: navigation, search


Development Sequence

Tasks for previous months (updated monthly)

September, 2011
October, 2011

IT Tasks

This is a page for non-development tasks. While not directly related to caret7 development, many of these tasks play a crucial role in our success:
IT Tasks

Procedure for Development

  1. All developers should add this page to their watch list so that they are alerted to any additional items or changes. Others are encouraged to sign up as well.
  2. When taking a task, put your inititials, an ETC (estimated time of completion), and status for that item (either in-progress or DONE).
  3. Please do not take a task that another developer is working on without following the suggestions below. Additionally, do not start a task before indicating that you are taking it.
  4. If an item's requirements, design, or implementation are unclear or controversial, then create an RT ticket, and put a link to it in the particular wiki item. For example, EventManager will require us to form a consensus, which is difficult to do with a wiki page, but easy to do by having a discussion on RT.
  5. If the design of a group of features (such as the names of objects, or GUI design) are un-decided, then an RT ticket should be clear labeled and put under the Design issues heading. This will facilitate the discussion of design related issues outside of meetings and allow us to better coordinate our efforts.
  6. In general, unless an item is a very large task, one developer should perform the implementation, and any concerns about how it is being done should be discussed through the wiki using the discussion page.
  7. Opening an RT ticket related to a development item is possible at any time, including after a task is marked DONE. If there are issues with an item, discussion is encouraged.
  8. At the beginning of each month, completed tasks should be MOVED to a separate page of tasks for the previous month (i.e. DevelopmentStrategy092011), and deleted from the "current tasks" list. Tasks started in the previous month(s), but that aren't finished, should be COPIED to the tasks for previous month, but still kept in the "current tasks" list. (Keeping tasks marked in-progress in both the previous month's tasks list and "current tasks" list should make it easier to spot tasks that took more than one month to complete).

Development Environment

  1. 64-bit only as a goal. We will be doing 32-bit builds in house currently, and will test user reaction in October. If needed we can do a 32 bit release, or drop 32 bit support completely at that point.
  2. A build system that runs as files are checked into source control needs to be created. If there compilation or testing errors, emails are sent to developers.
  3. Investigate CTest, CDash, and CPack for use with build system. - TSC: CTest is now in use, but not connected to a dashboard
  4. Examine CppTest for unit testing.
  5. Minimize usage of third party libraries and their components. When possible, encapsulate code from the third party so that users of the class never see the third party library. For example, the base class for XML parsing is abstract. A concrete class for XML parsing is implemented using QT's XML parsing library. Users call a factory method in abstract XML parser class that returns a parsers. Delegates that receive the SAX XML tags and characters have no knowledge about the actual implementation of the XML parser. If fact, the QT XML parser could be replaced with a different XML parser and the delegates will not see any difference.

Design and Development Issues

  1. Begin design of graphical-user interface. Create mockup user interfaces using drawings or QtDesigner/QtQuick. Examine Caret5 and Caret6. Determine what MUST be in the user-interface. What can be moved to the command line. Compare Caret5's many menus to Caret6's operation dialogs.
  2. Should Mac version be more Mac-Like? Mac windows normally do not include a close button and are closed by the red circle in the top left corner.
  3. Begin creating a list of commands for command line processing.
  4. Begin writing user documentation.
  5. Icons
  6. File Naming Convention

Coding (For November, 2011)

Carryover from Sept, 2011

  1. (Gifti, Jon, ETC:9/2/2011, Status:In-Progrss, NOTE from JWH: These items have been separated and are in GIFTI directory) Separate read of MetaData, LabelTable, and other elements that are present in multiple files. These implementation should be moved to the NIFTI library so that they can be used by NIFTI, GIFTI, CIFTI, and Caret XML files.
  2. (Nifti Volume File Extension, moved to Nifti Reader Writer status update below)....
  3. (Files, Caret XML, Caret6) SpecFile (is there a better name, DataSet, DataInventory, DataFileList??? Also write its SAXReader. For each surface-based data file listed, there will be an attribute that identifies its structure. UPDATE by JWH: Should a 'SpecFile' be a standardized file like the '*IFTI' files? Does this relate to what Dan presented, the CFF file?

Carryover from October, 2011

  1. (Nifti, Caret5, Jon, Status: in development)
    1. Nifti Extension Reader/Writer (Jon, Status:On hold until after SFN) - currently reads and writes existing extensions, but is unable to parse or interpret extension data.
  2. (NiftiHelper, Caret5, Jon, Status:On hold until after SFN) - port NiftiHelper from caret5 to caret7. Nifti helper includes functions for interpreting Nifti matrix scaling, orientation, and center using the NiftiHeader supplied sform and qform matrices. It also includes conversion routines for sform and qform orientation matrices stored in the Nifti Header file. This extra level of interpretation will be necessary for caret7 to make use of volumes laid out in non-standard (non-LPI) orientation.
  3. (Cifti, Jon, Status: in development) Rewrite Cifti to use Nifti Header Reader/Writer and NiftiMatrix reader so that they both share a common code-base. Ideally, Cifti will inherit interface from NiftiFile so that they share a common interface.
  4. (Cifti, Jon, Status: taken) Add helper routines to Cifti to create sensible defaults for time series and connectivity files. This should reduce errors when creating files from scratch, and make it quite a bit easier to work with. Add other helper function to Cifti as needed.
  5. (Brain, Caret6 John, Status: Done) Brain will organize all data loaded into Caret. It will contain multiple brain structures. Files that are structure independent are kept within the Brain. Multiple subjects, which require a one spec file per subject, will require a SessionManager.
  6. (Brain, Caret6 John, Status: Done) BrainStructure will contain data for a specific brain structure. At this time it is used for surface-based data related to the cerebral cortices and the cerebellum. In the future, it could contain models of sub-cortical structures. Files are associated with a brain structure by the structure attribute and the number of nodes. The brain structure will contain all of its surface-based files.
  7. (Brain, Caret6 John, Status: Done) Surface extends the SurfaceFile class and contains methods for some surface operations.
  8. (Brain, Caret6) DisplaySettings (perhaps DataController) and its subclasses are needed to manage the display status of different data types.
  9. (Brain, Caret6/New, John, Status: Done) Controllers (formerly Viewers) control the display of surfaces, volumes, etc. in the Main and Viewing Window graphics areas.
  10. (Brain, Caret6, Done, 12 Oct 2011 JWH) DataFileReader Uses Files project classes to read a file. Note: 'Brain' reads data files.
  11. (Brain, Caret6, Done, 12 Oct 2011, JWH) SpecFileReader Reads the selected files into Caret from a SpecFile. Note: 'Brain' reads the selected files in a Spec File.
  12. (Brain, New, Taken, JS) Develop a container format for caret 'projects'. This could include using CFF as Dan Marcus mentioned, cdb, sqlite, or any other number of approaches. Since this also may tie into how we manage caret projects with spec files, I will attempt to coordinate this with JWH, among others.
  13. (Brain, CIFTI, Files, Caret5/6/New, Status: Taken, Jon) Connectivity data loading and processing. A bit more clarity needs to be added to describe what kinds of processing we want (cifti average to start?, should this be located in Brain?)
  14. (GuiQt, New John, Status: Done) Main Window, toolbar, and menus.
  15. (GuiQt, Done, JWH) Viewing Windows and yoking.
  16. (GuiQt, Caret5/New, Done, 12 Oct 2011, JWH) One file selection dialog for all files types (including SpecFile). It defaults to spec file for the filter. If additional information is needed, it appear in another dialog.
  17. (GuiQt, Caret5/6, John, Waiting on design completion) Manage Loaded Files Window.
  18. (GuiQt, Caret6/New) A multi-page window for use by Display Control and the various operation windows.
  19. (Brain, Caret6, Done, JWH) Volume Overlays.
  20. (Brain, Caret5/Caret6/New, In Work, 12 Oct 2011, JWH) OpenGL rendering, selection. Initially support fixed pipeline but provide interface that will allow programmable pipeline in the future.
  21. (Brain/GuiQt, In development, John) Create mouse and keyboard processing mechanism. Active one receives input events and provides labels for status bar.
  22. (GuiQt, Caret6/New) Display Control Window.
  23. (GuiQt. Caret5/6/New) Identification Window. Move out of toolbox?
  24. (GuiQt, New) Help system that may use Qt's WebKit implementation.
  25. (GuiQt, Brain, Files, Caret6/New, Tim/John, status: in progress) Interface to Connectome DataBase. Currently supports dense connectivity, should discuss how to communicate what type of cifti a URL is associated with.
  26. (GuiQt, New) Capture image(s) dialog.
  27. (GuiQt, New, JS) Animation system (rotation, time point sequencing, etc) JS - I spent roughly a year creating an animation toolkit at a previous job. Many of the concepts could carry over, however, interface design will likely involve JWH, Jenn, and Will.
  28. (Common or Animation, New, JS) This will handle the back-end work involved in calculating timelines, performing analysis of dense time series to highlight areas of interest on the timeline, creating smooth spatial rotation to different areas of interest as the time sequence is saved, etc. It will start very simple, but in the end could have quite a bit of power.) I think Animation is generic enough that it could be it's own library, or perhaps put in Common.
  29. (GuiQt, New) Extend toolbox to allow display of more than one toolbox at a time. Allow for putting them side by side at the bottom of the viewer to take advantage of wide format displays.
  30. (GuiQt, New) Design and implement a Dense Connectivity tab to the toolbox. Potential features may include saving and loading previously selected nodes, setting threshholds for what connections are displayed, controlling the palette that is used to display connectivity, etc.
  31. (GuiQt, New) Closely coupled with the Animation system. Add a Dense Time Series tab to the toolbox window. Potential interface elements to start would include a time line, pause, start, stop, rewind, fast-foward. Buttons could be added for loading a dense time series. The above interface elements don't even cover the minimum requirements for such a windows, but should highlight that it needs it's own space.

New Tasks

  1. (Caret 7, Status: taken, Jenn and Erin Priority: HIGH) User Documentation!!!
  2. (Caret 7, Status: New, Priority: HIGH, Done, John) Open file - add pulldown for recently selected directories.
  3. (Caret 7, Status: New, Priority: HIGH) On the palette popup, add several options:
    • Show the filename and map name that the current palette operates on.
    • Apply to all maps in selected file. This would apply the currently selected palette to all maps in the currently selected file
    • Apply to all 'On' files. This would apply the currently selected palette to all maps in all files that are 'On' in the current window.
    • Histogram. Discuss what we want displayed here.
  4. (Caret7 Build. Status: DONE, JS Priority: HIGH) - Complete implementation of build environment for caret7. This includes the following:
    • Create chroot build environments for linux based on ubuntu 8.04, consider testing environment for 6.04 if further backwards compatibility is needed
    • Install SFU (microsoft's system for unix) on window's build server, and get ssh server running. This will enable remote building for Windows builds. The previous strategy, of using Cygwin's ssh implementation, resulted in numerous issues with cmake.
  5. (Caret 7 Build, Status: DONE, JS Priority: HIGH) Build System – auto-build
  6. (Caret 7, Status: taken, JS, Priority: HIGH) Volume File Meta Data extensions
    • (HIGH) Emulate what’s in label, metric files
    • (LOW) Add other metadata links (e.g., Studies) [LOW; needs discussion – DVE, Donna, Matt, John H et al)
  7. (Caret 7, Status: taken, Jenn, Erin and JH? Priority: HIGH) Info box: Option to display info box separately on left/right or as popup
  8. (Caret 7, Status: taken, JH Priority: HIGH) Manage Loaded Files (viewing loaded files and saving modified files). Include Save As plus Save options (flexibility to change names and/or directories). (needs discussion – John H, Matt, DVE, others?)
  9. (Caret 7, Status: taken, JH Priority: HIGH) Complete implementation of Browser Window menus.
    • Includes convert open windows to tabs
  10. (Caret 7, Status: taken, TC Priority: HIGH) Interconversion between CIFTI and Matlab-compatible external binary
    • Both directions; need to avoid loss of metadata. [Tim C after discussion with Matt]
  11. (Caret 7, Status: taken,JH, Priority: MEDIUM/HIGH) Colorbar option on overlay panel - one for each overlay (put in Settings)
  12. (Caret 7 Build, Status: DONE, JS Priority: MEDIUM) build on check-in
  13. (Caret 7 SDK's Status: on-hold for now, will work on it as time allows JS Priority: MEDIUM/LOW) - Complete packaging of SDK's for each platform and put them in a standard location. Building SDK's for each platform will ensure that we are using the same build of QT, zlib, and openssh across our workstations. It will also help to eliminate the duplication of effort that occurs when we each have to set up our own build environments. Finishing the creation of automated scripts for updating the build-environment will make it much easier for us to stay up to date with the latest versions of 3rd party libraries. This is mostly done, but needs polishing.
  14. (Caret 7, Status:in progress, TC, Priority: MEDIUM) Command-line insertion of palette metadata into label, metric, CIFTI, volume files - metric palette command implemented, volume palette command implemented, but VolumeFile mapping metadata and writing volume file not currently implemented
    • (Low) Extend to other metadata types (provenance, etc.)
  15. (Caret 7 automated test suite, Status: New Priority: MEDIUM) - Finish implementing automated test suite that is executed nightly and sends updates to developers. Consider looking into cmake dashboard for monitoring tests.
  16. (Caret 7, Status: New, Priority: MEDIUM) Thresholding - Types and GUI
    • Incorporate slider bars. Explore dual-slider on one bar.
    • Discuss GUI preferences (Jenn, Erin, John H et al., initial implementation DONE)
  17. (Caret 7, Status: New Priority: MEDIUM) Palette Colorbar - Checkable option for each overlay on overlay panel?
  18. (Caret 7, Status: New Priority: MEDIUM) Palette Editing - Show over toolbox with 'back' button.
    • On the palette popup, add two options:
    • Show the filename and map name that the current palette operates on.
    • Apply to all maps in selected file.  This would apply the currently selected palette to all maps in the currently selected file
    • Apply to all 'On' files.  This would apply the currently selected palette to all maps in all files that are 'On' in the current window.
  19. (Caret 7, Status: taken, JS Priority: LOW/MEDIUM) Histogram display control
  20. (Caret 7, Status: New Priority: MEDIUM) Slider bars for movies and other displays.
  21. (Caret 7, Status: taken, TC Priority: MEDIUM) Interconversion between gifti, nifti, and cifti files. (especially left-right cerebral gifti into cifti and back). [Tim C after discussion with Matt, DVE]
  22. (Caret 7, Status: taken, JH Priority: MEDIUM, Done, John) Basic yoking
  23. (Caret 7, Status: New Priority: HIGH) Connectivity: Add option (in connectivity tab?) to disable loading of dense connectivity data when a node is clicked so that values can be explored through identification operations.
  24. (Caret 7, Status: taken, JH Priority: MEDIUM) Allow more reduction in main window width
  25. (Caret 7, Status: taken, JS Priority: MEDIUM) Connectivity: Timepoint spin box, show both index and time but enter time.
  26. (Caret 7, Status: taken, JS Priority: MEDIUM) Connectivity: Display time-sequence for a node/voxel in a chart.
  27. (Caret 7, Status: taken, JH Priority: HIGH) Finish Spec File Dialog. (John H after discussing with Jenn, Erin, Matt, DVE)
  28. (Caret 7, Status: taken, JH Priority: HIGH) Border viewing (John H after discussing with Jenn, Erin, Matt, DVE)
  29. (Caret 7, Status: DONE, TC Priority: MEDIUM) Metric smoothing for irregular meshes (GEO_GAUSS_AREA)
  30. (Caret 7, Status: New Priority: MEDIUM) Assign borders to structure, as with metric, label
    • But add option to view borders on a compatible model (e.g., left borders on right hem) (needs discussion)
  31. (Caret 7, Status: New Priority: MEDIUM) Draw borders, fill in ROIs. (discuss with Jenn, Erin, Matt, DVE)
  32. (Caret 7, Status: New Priority: MEDIUM) Draw border update
  33. (Caret 7, Status: New Priority: MEDIUM Option to draw borders by connecting highlighted nodes (with spline fit?)
  34. (Caret 7, Status: New Priority: MEDIUM) Scene files.
    • file-specific (MEDIUM);
    • generic (LOW)
  35. (Caret 7, Status: DONE, TC Priority: MEDIUM) Volume to surface mapping, enclosing voxel, trilinear interpolated, and ribbon constrained are implemented - cubic spline dropped for now (FSL's cubic splines are quite involved)
  36. (Caret 7, Status: New Priority: MEDIUM) Tractography visualization (see document from last February)
  37. (Caret7, status: New Priority: MEDIUM) Connectivity: Add option to disable loading of dense connectivity data when a node is clicked so that values can be explored through identification operations.
  38. (Caret7, status: New Priority: MEDIUM) Connectivity: Timepoint spin box, show both index and time but enter time.
  39. (Caret7, status: New Priority: MEDIUM) Connectivity: Display time-sequence for a node/voxel in a chart.
  40. (Caret 7, Status: New Priority: LOW) Fancy yoking [Jenn, Will et al. lead discussion once basic yoking works]
  41. (Caret 7, Status: New Priority: LOW) Foci
  42. (Caret 7, Status: New Priority: LOW) Metadata - comments, naming maps, studies, vocabulary, ontology, etc. [ DVE lead discussion in early December]

Future Tasks

  1. (Common, Partially implemented but no GUI, John, Status: Done) Preferences - This will likely encapsulate Qt's preferences class.
  2. (Files, GIFTI-based) SurfaceRegionOfInterest
  3. (Command Line, Tim, Status: started) Begin writing command line operations. Architecture and templates for new commands ready for use.
  4. (Command Line) Command that converts a coordinate and a topology file to a surface file.
  5. (GuiQt) Metric Operations Window (uses multi-page windows).
  6. (GuiQt) Label Operations Window (uses multi-page windows).
  7. (GuiQt) Shape Operations Window (uses multi-page windows).
  8. (GuiQt) Surface Region of Interest Window (Wizard)
  9. (GuiQT) Generic gui interface to operations/algorithms (would replace many separate gui tasks specific to single operations)
  10. (GuiQt) BrainTips.
  11. (Files, Caret XML) The coordinate of border point or a focus is one or more of stereotaxic (cartesian), barycentric, or "van essen". As a result, a common object that supports each of these coordinates is extended or encapsulated. When the coordinate is stereotaxic, the file is essentially a Caret5 BorderFile or FociFile. In addition this object will also include a structure attributes. When a focus projects ambiguously, it will contain two or more of these objects.
  12. (Files, Caret XML) BorderProjection and its SAXReader.
  13. (Files, Caret XML) FociProjection and its SAXReader.
  14. (Files) Point location algorithm.
  15. (Files) Point projection algorithm.
  16. (Files) Algorithm SurfaceProjector for Foci and Borders.
  17. (Files) Algorithm SurfaceUnprojector for Foci and Borders.
  18. (Files, Caret XML) Annotation and its SAXReader.
  19. (Files, Caret XML) FociSearch and its SAXReader.
  20. (Files, Caret XML) Study (and its related elements) and its SAXReader.
  21. (Files, Caret XML) StudyCollection and its SAXReader.
  22. (GuiQt) Border Operations Window (uses multi-page windows).
  23. (GuiQt) Foci Operations Window (uses multi-page windows).
  24. (GuiQt) Study Operations Window (uses multi-page windows).
  25. (GuiQt) Vocabulary Operations Window (uses multi-page windows).
  26. (Brain) Mapping Surface Data to Volumes.
  27. (GuiQt) Gui for mapping surface data to volumes.
  28. (GuiQt) Gui for Mapping Volume (Atlas) Data to Surfaces.
  29. (GuiQt) Gui for Mapping Volume (Individual) Data to Surfaces.
  30. (Brain) Multiresolution morphing.
  31. (Brain) Surface-based Registration.
  32. (GuiQt) Geometry Operations Window.
  33. (GuiQt) Topology Operations Window.
  34. (GuiQt) Registration Operations Window.
  35. (Gifti) For statistics randomization, a special GIFTI reader that allows sequential reading of data arrays is needed. This means that after an array is read, the data is passed to a callback or interface method that processes the data. These operations are repeated until all data has been read. Similar functionality is also needed for writing of GIFTI files.
  36. (Statistics) Implement inferential and significance statistical algorithms.
  37. (Statistics) Use GLM for inferential statistics?
  38. (Caret5) Add ability or caret_command (carert5) to write Caret6 format data files including spec file.
  39. (Files) Add ability to read legacy file formats. Possibly command line only.
  40. SureFit an optional library that may require VTK.
  41. Get nightly builds and tests running consistently on all platforms. Continue to standardize development environment.
  42. Before releasing workbench to the public take advantage of this opportunity to create official time windows of support for each release, that give the user a set of guidelines for when they will need to upgrade their systems to continue using the latest workbench features (i.e. If you want to run workbench, you will need to use this release or later of your favorite OS). Enacting a well-defined support window (or life-cycle) will greatly reduce time wasted tracking down arcane and esoteric problems compatibility problmes that take a great deal of time to fix but benefits only a very small set of users.


  • Command classes should be callable from within Caret7 (as in Caret6).
  • Encapsulate Qt when possible.
  • Do not use QThread, use OpenMP.
Personal tools
Sums Database