Skychart

User Tools

Site Tools

Trace: • catgen
en:documentation:catgen

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
en:documentation:catgen [2009/01/12 23:40] – marc_fokkeren:documentation:catgen [2016/09/24 17:21] (current) – [Notes for Linux and Mac users] pch
Line 1: Line 1:
 ====== CATGEN ====== ====== CATGEN ======
  
-FIXME Please, correct my poor english :-\+From the menu: **Setup -> Catalog**, then in the dialog, click the **CatGen button** on the right. 
 +\\ \\ 
 +You can download thousands of Catalog text files with data of any kind of celestial objects from the **[[http://cdsweb.u-strasbg.fr/cats/Cats.htx|CDS]]** or **[[http://adc.gsfc.nasa.gov/|ADC]]** web site, and other sources. Each one contains the information about a few tens to a many millions of objects. 
  
-The catalog text files with data of any kind of celestial objects are available by the thousands from the **[[http://cdsweb.u-strasbg.fr/cats/Cats.htx|CDS]]** or **[[http://adc.gsfc.nasa.gov/|ADC]]** web site, or from other sources. Each one contains the information about a few tens to a many millions of objects.  +The objective of this program tool is to convert an ASCII catalog text file to a file that SkyCharts can use efficiently. Due to the large amount of available data, it will be impossible to make a choice that will fit all your needs. When you have a specific interest, you can use CatGen to build your own data set from the original professional catalog data. You also can use any new catalog without delay, as soon as it is published.
- +
-The objective of this program tool is to prepare a catalog text file for an efficient use with the SkyCharts software. +
-Due to the large amount of available data, it is impossible to make a choice that will fill all your needs. When you have a specific interest, you can use CatGen to build your own data set from the original professional catalog data. You also can use any new catalog without delay, as soon as it is published.+
  
 For a good performance of your newly built catalog in SkyCharts, you can use CatGen to convert the original text based catalog to a catalog in an organized binary format.  For a good performance of your newly built catalog in SkyCharts, you can use CatGen to convert the original text based catalog to a catalog in an organized binary format. 
Line 28: Line 27:
     26- 31  I6     ---       HD         [1/225300]? Henry Draper Catalog Number     26- 31  I6     ---       HD         [1/225300]? Henry Draper Catalog Number
  
-It is important to be familiar with the format of the text based catalog before you try to convert it with CatGen. There are lots of details to pay attention at: units, value ranges (between square brackets), number formats, identifiers, coordinate formats and epochs. A question mark in the Explanations indicate that this data is not always present, look at the data file itself to have an idea of it's frequency. Also be sure to read the notes, when available. A file editor that can handle large files (like **[[http://www.simtel.net/pub/dl/11983.shtml|PFE]]**) is very useful at this step.+It is important to be familiar with the format of the text based catalog before you try to convert it with CatGen. There are lots of details to pay attention at: units, value ranges (between square brackets), number formats, identifiers, coordinate formats and epochs. A question mark in the Explanations indicate that this data is not always present, look at the data file itself to have an idea of it's frequency. Also be sure to read the notes, when available. A file editor that can handle large files (like **[[http://notepad-plus-plus.org/|Notepad++]]**) is very useful at this step.
  
-If the data only is available in another format, it is generally easy to write a simple program to extract and format them.  A scripting language like Perl can be very useful for that. To handle the CSV files you can use the very good **[[http://home.hccnet.nl/s.j.francke/csvdb/csvdb.zip|CSVdb]]**, or **Calc** included with **[[http://www.openoffice.org/|OpenOffice]]**.+If the data only is available in another format, it is generally easy to write a simple program to extract and format them.  A scripting language like Perl can be very useful for that. To handle the CSV files you can use the very good **[[http://home.hccnet.nl/s.j.francke/csved/csvedsetup.exe|CSVed]]**, or **Calc** included with **[[http://www.openoffice.org/|OpenOffice]]**.
  
 You have to select which data to include in your catalog version. For each catalog type, there is a minimal quantity of information that is required to draw the object on the chart. But you can choose to add a number of the other fields with information to display, when you click on such an object. You have to select which data to include in your catalog version. For each catalog type, there is a minimal quantity of information that is required to draw the object on the chart. But you can choose to add a number of the other fields with information to display, when you click on such an object.
  
 Remember, the more information you add to the catalog, the more file space will be used on your computer hard disk. Remember, the more information you add to the catalog, the more file space will be used on your computer hard disk.
-Thus it's not always a good idea to add all the available information, take some time to know what you really need. You can also choose to build two catalog versions, one complete for the desktop computer, and one compact with minimal information for the laptop with limited disk space.+It's not allways a good idea to add all the available information, take some time to know what you really need. You can also choose to build two catalog versions, one complete for the desktop computer, and one compact with minimal information for the laptop with limited disk space.
  
  
Line 50: Line 49:
   * **Save Project** allows you to create (if it didn't already exist) the "xxx.prj" file (xxx = catalog short name).   * **Save Project** allows you to create (if it didn't already exist) the "xxx.prj" file (xxx = catalog short name).
   * **Load project** restores the ".prj" file of your choice.   * **Load project** restores the ".prj" file of your choice.
 +
 +
  
  
Line 56: Line 57:
 ===== Page 1 ===== ===== Page 1 =====
 {{  en:documentation:catgen_1_dsl.jpg|}} {{  en:documentation:catgen_1_dsl.jpg|}}
 +
 In the first page you need to indicate the **Catalog type** in your catalog, this activates some specific options in the next pages. In the first page you need to indicate the **Catalog type** in your catalog, this activates some specific options in the next pages.
  
Line 64: Line 66:
 Click the **Open** button to select the catalog text file. The large catalogs are often split in many smaller files, in such a case select all the files at the same time, the file order doesn't matter. Click the **Open** button to select the catalog text file. The large catalogs are often split in many smaller files, in such a case select all the files at the same time, the file order doesn't matter.
  
-Choose **Output Catalog Type**. With SkyChart V3, you can choose text file output for small catalogs.+Choose **Output Catalog Type**. With SkyChart V3, you can choose text file for small catalogs. In this case no data is written out but just a file descriptor that let the program to access your original text file. This way a change to the file is reported to the chart without having to run Catgen again. 
 + 
 +If you select "Text file catalog" you also have the option to specify an "Update URL" that point to the latest version of the file on the Internet. This way you can get the last version by a single click in the catalog setup.
  
 Click the **Next** button to go to the second page. Click the **Next** button to go to the second page.
-\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ +\\ \\ \\   
 + 
 + 
  
  
Line 76: Line 83:
 ===== Page 2 ===== ===== Page 2 =====
 {{  en:documentation:catgen_2_ol.jpg|}} {{  en:documentation:catgen_2_ol.jpg|}}
 +
 Select the input format of the coordinates: Select the input format of the coordinates:
   * **RA options** Hour, minutes, seconds or decimal Hour or Degrees, minutes, seconds or decimal degrees.   * **RA options** Hour, minutes, seconds or decimal Hour or Degrees, minutes, seconds or decimal degrees.
   * **DEC options** Degrees, minutes, seconds or decimal degrees or South Polar Distance.   * **DEC options** Degrees, minutes, seconds or decimal degrees or South Polar Distance.
-**Coordinates Equinox** is the epoch of the coordinates related to the precession. Use 2000 for ICRS.\\ +**Coordinates Equinox** is the epoch of the coordinates related to the precession. Use 2000 for **[[http://aa.usno.navy.mil/faq/docs/ICRS_doc.php|ICRS]]**.\\ 
 **Position Epoch** is the default epoch position for the proper motion, this date can be changed for each object later in the data file.\\  **Position Epoch** is the default epoch position for the proper motion, this date can be changed for each object later in the data file.\\ 
 **Maximum magnitude** is the magnitude of the faintest object in this catalog. This is particularly important for the stars catalog to properly set the magnitude scale on the chart. **Maximum magnitude** is the magnitude of the faintest object in this catalog. This is particularly important for the stars catalog to properly set the magnitude scale on the chart.
  
-The second part depends on the catalog type. For stars it is empty. the picture at the right hand side shows **Default Outline parameters** options:+The second part depends on the catalog type. For stars it is empty. The picture at the right hand side shows **Default Outline parameters** options:
   * **Drawing type** (used to connect points in the list) :   * **Drawing type** (used to connect points in the list) :
-    * Line : straight lines  +    * Line: straight lines  
-    * Spline : spline curves +    * Spline: spline curves 
-    * Surface : fill surface with line color +    * Surface: fill surface with line color 
-  * **Closed contour** force a closing line between last point and first one.+  * **Closed contour** force a closing line between the last point and the first one.
   * **Line width** as it says (in pixels)    * **Line width** as it says (in pixels) 
   * **Color** of the line or surface (click on to change it)   * **Color** of the line or surface (click on to change it)
-  * **Edit line operation** (see below) set character strings that will be recognized for drawing operations (coma separated values). Click return when ready. +  * **Edit line operation** (see below) set character strings that will be recognized for drawing operations (comma separated values). Click return when ready. 
-  * **Edit color** (see below) set character strings that will be recognized for lines color (coma separated values). Click return when ready.+  * **Edit color** (see below) set character strings that will be recognized for lines color (comma separated values). Click return when ready. 
 {{en:documentation:catgen_2a_ol.jpg|}} {{en:documentation:catgen_2b_ol.jpg|}} {{en:documentation:catgen_2a_ol.jpg|}} {{en:documentation:catgen_2b_ol.jpg|}}
  
 **Nebulae Options**  **Nebulae Options** 
 {{  en:documentation:catgen_2_dsl.jpg|}} {{  en:documentation:catgen_2_dsl.jpg|}}
-  * **Dimension and Units** default dimensions (if missing data) and unit.+ 
 +  * **Dimension and Units** default dimensions (in case of empty fields: no data) and unit.
   * **Logaritmic scale** check this box if the field used for size is a flux, i.e. for a radio source catalog.   * **Logaritmic scale** check this box if the field used for size is a flux, i.e. for a radio source catalog.
-  * **Edit Object Type** set character strings that will be recognized to set the object type (coma separated values). Click return when ready. +  * **Edit Object Type** set character strings that will be recognized to set the object type (comma separated values). Click return when ready. 
-  * **Edit Units** set character strings that will be recognized to set the unit size (coma separated values). Click return when ready.+  * **Edit Units** set character strings that will be recognized to set the unit size (comma separated values). Click return when ready. 
 {{en:documentation:catgen_2a_dsl.jpg|}} {{en:documentation:catgen_2b_dsl.jpg|}} {{en:documentation:catgen_2a_dsl.jpg|}} {{en:documentation:catgen_2b_dsl.jpg|}}
  
-Click the Next button to go to the third page.+Click the **Next** button to go to the third page. 
  
  
Line 153: Line 165:
 If you check **Use this field name as an Alternate name** the string value can be used as an alternate name for the object. This name is used in the case the main name (Catalog Id) is missing or it can be added to the index file of the catalog.\\  If you check **Use this field name as an Alternate name** the string value can be used as an alternate name for the object. This name is used in the case the main name (Catalog Id) is missing or it can be added to the index file of the catalog.\\ 
 {{  en:documentation:catgen_3_dsl_adv.jpg|}} {{  en:documentation:catgen_3_dsl_adv.jpg|}}
-**Advanced** button enables you to compute a linear transformation of the catalog data, it is active only for numeric values.\\  + 
-A use is to obtain the object coordinates when not directly available from the catalog.\\ +The **Advanced** button enables you to compute a linear transformation of the catalog data. It is active only for numeric values.\\  
 +A use can be to obtain the object coordinates when these aren't directly available from the catalog.\\ 
 If the catalog gives the object position in millimeters in the plate from the galaxy center (i.e. ra: 0h42m42.00s dec:+41°16'00.0") and you know the top of the plate is at North and the plate scale is 2 seconds per millimeter.\\  If the catalog gives the object position in millimeters in the plate from the galaxy center (i.e. ra: 0h42m42.00s dec:+41°16'00.0") and you know the top of the plate is at North and the plate scale is 2 seconds per millimeter.\\ 
 The approximate (without plate correction) coordinates in degrees are: The approximate (without plate correction) coordinates in degrees are:
Line 172: Line 185:
 ===== Page 4 ===== ===== Page 4 =====
 {{  en:documentation:catgen_4_dsl1.jpg|}} {{  en:documentation:catgen_4_dsl1.jpg|}}
 +
 In the last page you set the options for the generated catalog. In the last page you set the options for the generated catalog.
  
Line 186: Line 200:
  
 {{  en:documentation:catgen_prog_dsl.jpg?300|}} {{  en:documentation:catgen_prog_dsl.jpg?300|}}
 +
 A progress box shows you the current operation. You can interrupt the process by pressing the **Abort** button. In such a case the catalog will not be usable.\\  A progress box shows you the current operation. You can interrupt the process by pressing the **Abort** button. In such a case the catalog will not be usable.\\ 
 After the build is complete, the progress box is closed. You can exit the program with the **Close** button. After the build is complete, the progress box is closed. You can exit the program with the **Close** button.
Line 194: Line 209:
  
 If all worked well, you are now ready to use the catalog. Refer to **[[http://www.ap-i.net/skychart/en/documentation/catalog#catalog|SkyChart documentation]]** to activate the catalog. If all worked well, you are now ready to use the catalog. Refer to **[[http://www.ap-i.net/skychart/en/documentation/catalog#catalog|SkyChart documentation]]** to activate the catalog.
 +
 +===== Notes for Linux and Mac users =====
 +
 +Please read carefully if you plan to make a big catalog with 9537 files.
 +
 +As it read lines from the input files Catgen need to randomly write the data to
 +the 9537 output files, depending of the star coordinates. So the 9537 files need
 +to be open at the same time.
 +
 +As Linux and Mac OS are multi-user operating system there is safe-guard to avoid one user
 +to eat all of the system resource. For a standard Linux installation for example the
 +maximum number of file a user can open at the same time is limited to 1024.
 +
 +On Mac OS X the way to do may vary depending on your version. See the first answer [[http://unix.stackexchange.com/questions/108174/how-to-persist-ulimit-settings-in-osx-mavericks|to this question]].
 +
 +On Linux you first need to change the system limit in the file
 +limits.conf :
 +<code>
 +sudo vi /etc/security/limits.conf
 +</code>
 +add this two lines at the end of the file:
 +<file>
 +* soft nofile 1024
 +* hard nofile 32768
 +</file>
 +I recommend you keep the default value of 1024, this can be a good protection
 +again a runaway process.
 +
 +Save the file, and logout from the system as this file only apply at next login.
 +
 +Then open a terminal window and type : 
 +<code>
 +ulimit -S -n 20000
 +</code>
 +and from the same terminal type : 
 +<code>
 +skychart
 +</code>
 +This is very important as the limit change is only for the current instance of
 +the shell, not system width.
  
en/documentation/catgen.1231800032.txt.gz · Last modified: 2015/11/06 20:35 (external edit)