Skychart

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== CATGEN ======

From the menu: **Setup -> Catalog**, then in the dialog, click the **CatGen button** on the right.
\\ \\
You can download thousands of Catalog text files with data of any kind of celestial objects from the **[[http://cdsweb.u-strasbg.fr/cats/Cats.htx|CDS]]** or **[[http://adc.gsfc.nasa.gov/|ADC]]** web site, and other sources. Each one contains the information about a few tens to a many millions of objects. 

The objective of this program tool is to convert an ASCII catalog text file to a file that SkyCharts can use efficiently. Due to the large amount of available data, it will be impossible to make a choice that will fit all your needs. When you have a specific interest, you can use CatGen to build your own data set from the original professional catalog data. You also can use any new catalog without delay, as soon as it is published.

For a good performance of your newly built catalog in SkyCharts, you can use CatGen to convert the original text based catalog to a catalog in an organized binary format. 

Generally, a catalog is a simple formatted text file. In such a catalog, every single line (a record) contains all the data for one individual object. The data of each line consists of separate pieces (fields) of information. These contain the identifier of the object in the catalog, and also determine the properties of this object. The content of every separate piece of data for every line is simply defined by a column position and a length. For example, consider this:

   1          BD+44 4550      3 36042          46           000001.1+444022000509.9+451345114.44
   2          BD-01 4525      6128569                       235956.2-010330000503.8-003011 98.33
   3 33    PscBD-06 6357     281285721002I         Var?     000013.0-061601000520.1-054227 93.75
   4 86    PegBD+12 5063     87 917012004                   000033.8+125023000542.0+132346106.19

At certain fixed positions, there is a fixed type of information. Usually a companion file (Readme) describes the file format with all the necessary details. Detailed information about this file is **[[http://vizier.u-strasbg.fr/doc/catstd.htx|here]]** .

  Byte-by-byte Description of file: catalog.dat
  --------------------------------------------------------------------------------
     Bytes Format  Units     Label      Explanations
  --------------------------------------------------------------------------------
     1-  4  I4     ---       HR         [1/9110]+ Harvard Revised Number = Bright Star Number
     5- 14  A10    ---       Name       Name, generally Bayer and/or Flamsteed name
    15- 25  A11    ---       DM         Durchmusterung Identification (zone in bytes 17-19)
    26- 31  I6     ---       HD         [1/225300]? Henry Draper Catalog Number

It is important to be familiar with the format of the text based catalog before you try to convert it with CatGen. There are lots of details to pay attention at: units, value ranges (between square brackets), number formats, identifiers, coordinate formats and epochs. A question mark in the Explanations indicate that this data is not always present, look at the data file itself to have an idea of it's frequency. Also be sure to read the notes, when available. A file editor that can handle large files (like **[[http://notepad-plus-plus.org/|Notepad++]]**) is very useful at this step.

If the data only is available in another format, it is generally easy to write a simple program to extract and format them.  A scripting language like Perl can be very useful for that. To handle the CSV files you can use the very good **[[http://home.hccnet.nl/s.j.francke/csved/csvedsetup.exe|CSVed]]**, or **Calc** included with **[[http://www.openoffice.org/|OpenOffice]]**.

You have to select which data to include in your catalog version. For each catalog type, there is a minimal quantity of information that is required to draw the object on the chart. But you can choose to add a number of the other fields with information to display, when you click on such an object.

Remember, the more information you add to the catalog, the more file space will be used on your computer hard disk.
It's not allways a good idea to add all the available information, take some time to know what you really need. You can also choose to build two catalog versions, one complete for the desktop computer, and one compact with minimal information for the laptop with limited disk space.



===== CATGEN Structure =====

CATGEN has four steps, each on one page:
  * **Page 1** Select the input file, name your catalog, and set the catalog type
  * **Page 2** Set the general parameters according to the catalog type
  * **Page 3** Define the fields to read from the input file to get them included in your catalog
  * **Page 4** Set the output options for your catalog and build it
At each step, you can save your project in a **".prj"** file. This contains all the settings you made in CatGen that determine how you want to convert the input file to your catalog. So, you can interrupt building the catalog and continue it later on after loading your .prj file.\\ 
The two buttons at bottom of every page in CatGen manage this facility:
  * **Save Project** allows you to create (if it didn't already exist) the "xxx.prj" file (xxx = catalog short name).
  * **Load project** restores the ".prj" file of your choice.






===== Page 1 =====
{{  en:documentation:catgen_1_dsl.jpg|}}

In the first page you need to indicate the **Catalog type** in your catalog, this activates some specific options in the next pages.

The **Catalog short name** is used to identify the catalog. The shortname will be prefixed to the object name in the charts. It must be one to four character long.

The **Catalog long name** is the catalog full identifier.

Click the **Open** button to select the catalog text file. The large catalogs are often split in many smaller files, in such a case select all the files at the same time, the file order doesn't matter.

Choose **Output Catalog Type**. With SkyChart V3, you can choose text file for small catalogs. In this case no data is written out but just a file descriptor that let the program to access your original text file. This way a change to the file is reported to the chart without having to run Catgen again.

If you select "Text file catalog" you also have the option to specify an "Update URL" that point to the latest version of the file on the Internet. This way you can get the last version by a single click in the catalog setup.

Click the **Next** button to go to the second page.
\\ \\ \\  









===== Page 2 =====
{{  en:documentation:catgen_2_ol.jpg|}}

Select the input format of the coordinates:
  * **RA options** Hour, minutes, seconds or decimal Hour or Degrees, minutes, seconds or decimal degrees.
  * **DEC options** Degrees, minutes, seconds or decimal degrees or South Polar Distance.
**Coordinates Equinox** is the epoch of the coordinates related to the precession. Use 2000 for **[[http://aa.usno.navy.mil/faq/docs/ICRS_doc.php|ICRS]]**.\\ 
**Position Epoch** is the default epoch position for the proper motion, this date can be changed for each object later in the data file.\\ 
**Maximum magnitude** is the magnitude of the faintest object in this catalog. This is particularly important for the stars catalog to properly set the magnitude scale on the chart.

The second part depends on the catalog type. For stars it is empty. The picture at the right hand side shows **Default Outline parameters** options:
  * **Drawing type** (used to connect points in the list) :
    * Line: straight lines 
    * Spline: spline curves
    * Surface: fill surface with line color
  * **Closed contour** force a closing line between the last point and the first one.
  * **Line width** as it says (in pixels) 
  * **Color** of the line or surface (click on to change it)
  * **Edit line operation** (see below) set character strings that will be recognized for drawing operations (comma separated values). Click return when ready.
  * **Edit color** (see below) set character strings that will be recognized for lines color (comma separated values). Click return when ready.

{{en:documentation:catgen_2a_ol.jpg|}} {{en:documentation:catgen_2b_ol.jpg|}}

**Nebulae Options** 
{{  en:documentation:catgen_2_dsl.jpg|}}

  * **Dimension and Units** default dimensions (in case of empty fields: no data) and unit.
  * **Logaritmic scale** check this box if the field used for size is a flux, i.e. for a radio source catalog.
  * **Edit Object Type** set character strings that will be recognized to set the object type (comma separated values). Click return when ready.
  * **Edit Units** set character strings that will be recognized to set the unit size (comma separated values). Click return when ready.

{{en:documentation:catgen_2a_dsl.jpg|}} {{en:documentation:catgen_2b_dsl.jpg|}}

Click the **Next** button to go to the third page.





===== Page 3 =====
{{  en:documentation:catgen_3_dsl.jpg|}}

This page enables you to map the data from the input file to your catalog file.
The list shows different fields, depending from the catalog type you chose in the first page.\\
Select each field you want to include in you catalog and define the position in the sample record.\\ 
Required fields are enclosed between square brackets "[]", required units are between brackets "()".\\ 

Indicate in **Label** the label you want to show for this field.
**First Char** and **Length** describe the position of the field for each line in the input file. You can type the value here (obtained from the catalog Readme file) or mark the data in the sample record with the mouse. When you use your mouse, be sure the field length you mark will be long enough to contain the longest string of field information in the catalog file.

**Field list for each catalog type:**

**Required fields:**
^ Stars ^ Variable Stars ^ Double Stars ^ Nebulae ^ Nebulae Outlines^
| RA | RA | RA | RA | RA|
| DEC | DEC | DEC | DEC | DEC|
| Magnitude (V) |   | Magn. comp. 1 |   | Line operation |
|   |   | Separation |   |   |

**Fields used to draw the charts:**
^ Stars ^ Variable Stars ^ Double Stars ^ Nebulae ^ Nebulae Outlines^
| Catalog ID | Catalog ID | Catalog ID | Catalog ID | Catalog ID |
| B-V | Magnitude Max. | Position angle | Nebula type | Line width |
| Proper motion RA | Magnitude Min. |   | Magnitude | Line color |
| Proper motion DEC | Magnitude code |   | Surface brightness | Drawing type |
| Position Epoch |   |   | Largest dimension |   |
| Magnitude B |   |   | Smallest dimension |   |
|   |   |   | Dimension unit |   |
|   |   |   | Position angle |   |

**Fields for information only:**
^ Stars ^ Variable Stars ^ Double Stars ^ Nebulae ^ Nebulae Outlines^
| Magnitude R | Period | Magn. comp. 2 | Radial velocity | Comment |
| Spectral class | Variable type | Epoch | Morphological class | String.. |
| Parallax | Maxima Epoch | Component name | Comment | Numeric.. |
| Comment | Rise Time | Spectral class comp. 1 | String.. |   |
| String.. | Spectral class | Spectral class comp. 2 | Numeric.. |   |
| Numeric.. | Comment | Comment |   |   |
|   | String.. | String.. |   |   |
|   | Numeric.. | Numeric.. |   |   |

At the bottom of the list you find ten strings and numeric values you can use freely for any data.\\
If you check **Use this field name as an Alternate name** the string value can be used as an alternate name for the object. This name is used in the case the main name (Catalog Id) is missing or it can be added to the index file of the catalog.\\ 
{{  en:documentation:catgen_3_dsl_adv.jpg|}}

The **Advanced** button enables you to compute a linear transformation of the catalog data. It is active only for numeric values.\\ 
A use can be to obtain the object coordinates when these aren't directly available from the catalog.\\ 
If the catalog gives the object position in millimeters in the plate from the galaxy center (i.e. ra: 0h42m42.00s dec:+41°16'00.0") and you know the top of the plate is at North and the plate scale is 2 seconds per millimeter.\\ 
The approximate (without plate correction) coordinates in degrees are:
  DEC = 41.26667 + Xmm * (2/3600)
  RA = 10.67500 - Ymm * (2/3600/cos(DEC))
Select "Decimal degree" for both RA and DEC, indicate the position in millimeters and set the "Advanced" value to:
  for RA : -0.00073837 and 10.67500
  for DEC : 0.00055555 and 41.26667

If the data requires some more complexe transformations you must do the transformations before running Catgen. A Perl script can solve almost any case. 

Click the Next button to go to the fourth page.




===== Page 4 =====
{{  en:documentation:catgen_4_dsl1.jpg|}}

In the last page you set the options for the generated catalog.

Set the number of files as recommended depending the number of objects in the catalog, this is very important to obtain good performance.\\ 
Check the **create a search index file** box if you want to create a search index to search objects in this catalog by their names.\\ 
Check the **Add the alternate names to the index** box if you want the fields you defined as alternate names to be included in the index.\\ 
Check the **Prefix the alternate names with their label** box if you want to prefix the alternate names with the label you set for the corresponding field. I.e. if a column of the input file list the UGC catalog number, you can set the index value to UGC7442 instead of 7442, which can be confusing.\\ 
Choose an **Output directory** to save the catalog files. It is a good habit to use a separate directory for each catalog. Ususally these directories are subdirectories of ''[SkyCharts installation directory]/cat/''\\ 
Check the **Append to an existing catalog** box if you want to append the data to an existing catalog of the same name in the same directory.\\ 
This is useful to create a single catalog from various sources. In this case the catalog structure must be exactly the same as the existing catalog, otherwise the data is lost, please make a backup before to try this option.

Now you're this far, save your job definition with the **Save Project** button, to enable the possibility of easy changes. Now you can press the **Build Catalog** button!
\\ \\ \\ \\ 

{{  en:documentation:catgen_prog_dsl.jpg?300|}}

A progress box shows you the current operation. You can interrupt the process by pressing the **Abort** button. In such a case the catalog will not be usable.\\ 
After the build is complete, the progress box is closed. You can exit the program with the **Close** button.

In case a catalog file record doesn't contain a valid numeric value for a field, this object will not be included in the catalog. Those cases find their way to a file called **reject.txt**.

It's a good idea first to test your definitions with a limited number of records to avoid a file consisting of millions of rejected lines! You can find a Windows equivalent of the Unix commands "head", "tail" and "grep" in a package **[[http://sourceforge.net/project/showfiles.php?group_id=9328&package_id=9393|here]]** that will help you to extract a few lines from a large file.

If all worked well, you are now ready to use the catalog. Refer to **[[http://www.ap-i.net/skychart/en/documentation/catalog#catalog|SkyChart documentation]]** to activate the catalog.

===== Notes for Linux and Mac users =====

Please read carefully if you plan to make a big catalog with 9537 files.

As it read lines from the input files Catgen need to randomly write the data to
the 9537 output files, depending of the star coordinates. So the 9537 files need
to be open at the same time.

As Linux and Mac OS are multi-user operating system there is safe-guard to avoid one user
to eat all of the system resource. For a standard Linux installation for example the
maximum number of file a user can open at the same time is limited to 1024.

On Mac OS X the way to do may vary depending on your version. See the first answer [[http://unix.stackexchange.com/questions/108174/how-to-persist-ulimit-settings-in-osx-mavericks|to this question]].

On Linux you first need to change the system limit in the file
limits.conf :
<code>
sudo vi /etc/security/limits.conf
</code>
add this two lines at the end of the file:
<file>
* soft nofile 1024
* hard nofile 32768
</file>
I recommend you keep the default value of 1024, this can be a good protection
again a runaway process.

Save the file, and logout from the system as this file only apply at next login.

Then open a terminal window and type : 
<code>
ulimit -S -n 20000
</code>
and from the same terminal type : 
<code>
skychart
</code>
This is very important as the limit change is only for the current instance of
the shell, not system width.