$Id: TODO 737 2010-06-04 18:35:18Z wojdyr $

---------- TODO list and tentative development plan  ----------------

* finish powder diffraction add-on (Pawley fitting)

* command info:
  - "info 12, fit" doesn't work, although "info fit, 12" works.
  - support quoted strings, e.g. info '2+2=', 2+2
  - use new lexer
  - "info peaks" should use info-numeric-format

* FFT data transformation using either KISS FFT or FFTW library,
  fft[-inv]-xx, where xx is re, im, amp, e.g. @n = fft-re @3 

* convolution and deconvolution of two datasets or a dataset with Gaussian:
  convolve @0 @1; convolve @0 2.3; deconvolve @0 3.4

* embed Lua interpreter

* make possible something like this: $min = {min(y if a) in @*}

* GUI: there is ugly vertical line (a part of the fitted curve) in points
  where the value is undefined (e.g. in ReadShockley function for x=0).

* GUI: when dataset is changed and the active function (i.e. the function
  in the bottom of the sidebar) is not in the new active dataset, the active
  function should also be changed (to the function that was active last time
  this dataset was active)

* GUI: saving background and reusing it for the next dataset (i.e. loading
  saved background, changing it a bit and substracting)

* add functions cot and coth

* GUI: redesign gradient dialog

* GUI: Data>Load should use default load dir (as Quick Load)

* [BUG] parsing problems: "a=a" => OK (should be error)  "a=ab" => Syntax error

* (GUI) plot horizontal/vertical lines 
     can be used to show theoretical peak positions in XRD
     (or to mark simulation restarts in data from MD)
     Syntax: 
             plot line x=42.2 label="foo" in @*
             plot line y=1000.3 label="foo" in @*
             plot no lines in @*

* (GUI) a simple (for user) way to interrupt fitting or script
  Requires two thread and thread-safety.

-------------->

* confidence intervals of model parameters (95%, 99%)
  http://www.boost.org/doc/libs/1_42_0/libs/math/doc/sf_and_dist/html/math_toolkit/dist/stat_tut/weg/st_eg/tut_mean_intervals.html

* ? confidence intervals for data

* data smoothing (Savitzky-Golay and/or other methods)

* GUI: refactor MainPlot - mouse event handling based on subscriptions,
       a clean interface to XOR-drawing 

* magic default parameters: if $default__name is defined: when functions 
  with parameter `name' are "guessed", this parameter is set to $default__name
  (e.g. user sets $default__shape=2.5 to have initial value of Pearson7.shape
  equal 2.5). (Similar to FFrame::get_global_parameters()).

* GUI: it should be possible to constrain two parameters of two peak 
     (using mouse and peak-top menu), at least to make them equal.

* GUI: on "Guess function" toolbar button: if initial values are not known,
  show a dialog asking for the values.

* add a new function category: sigmoids/steps (now function can be guessed as
  "peak" and "linear"). (GUI): drawing drafts for the new category.

--------------> 

* add alpha channel to toolbar icons

* (GUI) single dialog for configuring main plot (and also aux. plots) (?)
  - now in Configure axes / peak labels fonts are changed (but shouldn't) 
    even if Apply is not pressed 
  - option to change the width of the sum line (and other lines)
  - ? changing data point size and line separately for each dataset,

* unused variables should not slow down fitting

* support variables in range in data transformation: Y[$foo-1] = 0

------------------ 

* GUI: use wxGraphicsContext or Cairo for drawing plots with antialiasing

* GUI, enhancements for the script editor: 
       - "info ...> file" marked differently than not redirected info 
       - paths of files to be read should be checked 
       - ? search'n'replace button

* user defined functions:
      - more sophisticated simplifications of expressions and CSE
      - calculating limits, width, area, etc. for UDF
      - ?? optionally use LLVM for faster calculation of user-defined functions

* EMG function: how to calculate it? Now there are some problems with
  this function. There must be smarter way to implement it...
  perhaps using gsl_sf_log_erfc or exp(x*x)*erfc(x) from
  http://www.netlib.org/specfun/erf

* improve Voigt function (cut-tails, better approximation),
  there are dozens of papers on Voigt calculation, mostly in JQSRT.

* GUI: menu Functions:
  Functions > Peak-Find Info -> animation
  Functions > Multidata P-F Info -> window with list of parameters (Xview-like)

* ? box-constrained simple variables 

* ?fitting with enforcing valid function's parameters: 
    eg. in Pearson shape > 0.5 , shape < 1e9
    what if parameter is out of domain? change? disable function?

* ? cmd/GUI: plot peak-groups: ==> plot (%a+%b+%c)  

* ?GUI: Data Editor - left/right/top/bottom - magic keywords: user would be
       asked to select point/range/rectangle in plot

* ? more sophisticated algorithms for peak detection
    - compare methods from origin pkfind, peakfit and other programs
    - research papers: http://dx.doi.org/10.1016/j.chroma.2004.11.073
                       http://dx.doi.org/10.1002/cem.1005

* fitting: GA fitting options, 
  a way to "shake" parameters for L-M

* fitting:
	simulated annealing (it can replace LM shake_before option)
	perhaps also BFGS, conjugate gradients methods
	Rosenbrock Method:
	  http://www.applied-mathematics.net/optimization/rosenbrock.html

* fitting: investigate using a library for nonlinear fitting/optimization:
  I don't know about any library good enough to justify migration.  
  The list of libraries in this category:
    NLopt: http://ab-initio.mit.edu/wiki/index.php/NLopt
    COOOL: http://coool.mines.edu/ 
    GAUL: http://gaul.sourceforge.net/, 
    WNLIB: http://www.willnaylor.com/wnlib.html
    GSL and 3rd-party extensions to GSL,
    netlib: http://www.netlib.org/ 
    OOL: http://OOL.sf.net, 
    scitbx::lbfgs http://cctbx.sourceforge.net/
    http://sal.jyu.fi/B/3/index.shtml, 
    http://coin-or.org
    http://www-unix.mcs.anl.gov/otc/Guide/faq/nonlinear-programming-faq.html 
    levmar: http://www.ics.forth.gr/~lourakis/levmar/
    OpenOpt: http://openopt.org/


* fitting: robust fit - other error distributions then Gaussian, 
  eg. Lorenzian...; so called 'maximum-likelihood method' -- using Poisson
  error distribution.

* ? exporting formulae in MathML (when option formula-export-style == mathml)

* [BUG, won't be fixed]: program crashes on MS Win 9x when data contains NaN 
  (e.g. as a result of asin(x) x>1) and "Zoom all" is called.

-------  Anticipated upgrades  -----------

* consider migration to Boost::Spirit 2. Alternatives are: using antlr or
  hand-crafted parser (simple lexer could be based on boost.tokenizer).
  For data operations, one of math parsers could be used:
       muParser http://muparser.sourceforge.net/
        http://www.codeproject.com/KB/recipes/FastMathParser.aspx?msg=2942508
       Function Parser http://warp.povusers.org/FunctionParser/
     we would need at least support for user-defined function compilers
     (to have syntax "sum(y if 3<x<8)") and callbacks for arrays (x[n+3])
    another parser is GNU libmatheval
  some statements can be replaced with functions of embedded Lua

* when wxWidgets 3.x will be available in major Linux distributions:
  - open-file dialog (with preview) will use XyBrowser
  - file dialogs will be changed to take adventage of custom controls:
      * Choose log file: add "log also output" checkbox
      * Data > Export will be done in one step
      * "Export formula" dialog will have formula-export-style option
  - button '(un)lock' on variables pane (wxToggleBitmapButton)
  - Redesign Data Editor:  
       - use wxDataViewCtrl for spreadsheet.
       - the spreadsheet should be a separate dialog, and it should also 
	 be available in "Transformation Mgr" on clicking "Data>>" button
       - use wxDataViewCtrl instead of wxListCtrl
       - no extra dialog for adding transformation
       - remove "description" field, use comments in "code" instead
  - use wxDataViewCtrl for the lists at the sidebar,
    refactor ListWithColors::populate(), add sorting by clicking on column 
    header (use case: sort functions by centers)
  - use wxToggleBitmapButton buttons on sidebar>functions (buttons that make 
    hwhm and shape equal for all functions)

  Writing Qt GUI is also considered.

-------  Ideas for people, who would like to contribute:  -----------

- graphics: make PNG icons for toolbar. XPM format (now in use) doesn't support
   alpha channel, and the toolbar doesn't look good.

- write a function to calculate a Voigt/Faddeeva function, based on one 
  (or more) of papers published about it. I mean here an approximation more 
  exact that the standard Humlicek one. GSL would probably also accept
  such a function in C.
  http://sources.redhat.com/ml/gsl-discuss/2004-q4/msg00061.html

- see the note about EMG function above

- if you have some expertise in nonlinear optimization methods, it's possible
  to do what Gnu Scientific Library seems to plan: put a few least-squares
  fitting methods with similar interface into one library. 
  It can be part of fityk, a separate library or a part of GSL.

- Add alpha channel to toolbar icons. Fityk icons are usually modified GTK 
  stock icons. Alpha-channel was removed when converting to xpm format.
  Now I see icons with alpha-channel look much better (like menu icons,
  visible in GTK version only), but making the icons again takes time...

- (wxWidgets): On the sidebar FancyRealCtrl is used for changing parameters:
  http://wxforum.shadonet.com/viewtopic.php?t=13471
  It would be better to use a custom (3D?) control instead of wxSlider,
  perhaps similar to controls used in Google Earth for zoom and tilt.

- anything else

---------------------------------------------------------------------
