Skychart

User Tools

Site Tools

Trace:
en:documentation:catgen

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
en:documentation:catgen [2009/01/12 23:57] – marc_fokkeren:documentation:catgen [2016/09/24 17:21] (current) – [Notes for Linux and Mac users] pch
Line 1: Line 1:
 ====== CATGEN ====== ====== CATGEN ======
  
-FIXME Please, correct my poor english :-\+From the menu: **Setup -> Catalog**, then in the dialog, click the **CatGen button** on the right. 
 +\\ \\ 
 +You can download thousands of Catalog text files with data of any kind of celestial objects from the **[[http://cdsweb.u-strasbg.fr/cats/Cats.htx|CDS]]** or **[[http://adc.gsfc.nasa.gov/|ADC]]** web site, and other sources. Each one contains the information about a few tens to a many millions of objects. 
  
-Catalog text files with data of any kind of celestial objects are available by the thousands from the **[[http://cdsweb.u-strasbg.fr/cats/Cats.htx|CDS]]** or **[[http://adc.gsfc.nasa.gov/|ADC]]** web site, and other sources. Each one contains the information about a few tens to a many millions of objects.  +The objective of this program tool is to convert an ASCII catalog text file to a file that SkyCharts can use efficiently. Due to the large amount of available data, it will be impossible to make a choice that will fit all your needs. When you have a specific interest, you can use CatGen to build your own data set from the original professional catalog data. You also can use any new catalog without delay, as soon as it is published.
- +
-The objective of this program tool is to prepare a catalog text file for an efficient use with the SkyCharts software. +
-Due to the large amount of available data, it is impossible to make a choice that will fill all your needs. When you have a specific interest, you can use CatGen to build your own data set from the original professional catalog data. You also can use any new catalog without delay, as soon as it is published.+
  
 For a good performance of your newly built catalog in SkyCharts, you can use CatGen to convert the original text based catalog to a catalog in an organized binary format.  For a good performance of your newly built catalog in SkyCharts, you can use CatGen to convert the original text based catalog to a catalog in an organized binary format. 
Line 28: Line 27:
     26- 31  I6     ---       HD         [1/225300]? Henry Draper Catalog Number     26- 31  I6     ---       HD         [1/225300]? Henry Draper Catalog Number
  
-It is important to be familiar with the format of the text based catalog before you try to convert it with CatGen. There are lots of details to pay attention at: units, value ranges (between square brackets), number formats, identifiers, coordinate formats and epochs. A question mark in the Explanations indicate that this data is not always present, look at the data file itself to have an idea of it's frequency. Also be sure to read the notes, when available. A file editor that can handle large files (like **[[http://www.simtel.net/pub/dl/11983.shtml|PFE]]**) is very useful at this step.+It is important to be familiar with the format of the text based catalog before you try to convert it with CatGen. There are lots of details to pay attention at: units, value ranges (between square brackets), number formats, identifiers, coordinate formats and epochs. A question mark in the Explanations indicate that this data is not always present, look at the data file itself to have an idea of it's frequency. Also be sure to read the notes, when available. A file editor that can handle large files (like **[[http://notepad-plus-plus.org/|Notepad++]]**) is very useful at this step.
  
-If the data only is available in another format, it is generally easy to write a simple program to extract and format them.  A scripting language like Perl can be very useful for that. To handle the CSV files you can use the very good **[[http://home.hccnet.nl/s.j.francke/csvdb/csvdb.zip|CSVdb]]**, or **Calc** included with **[[http://www.openoffice.org/|OpenOffice]]**.+If the data only is available in another format, it is generally easy to write a simple program to extract and format them.  A scripting language like Perl can be very useful for that. To handle the CSV files you can use the very good **[[http://home.hccnet.nl/s.j.francke/csved/csvedsetup.exe|CSVed]]**, or **Calc** included with **[[http://www.openoffice.org/|OpenOffice]]**.
  
 You have to select which data to include in your catalog version. For each catalog type, there is a minimal quantity of information that is required to draw the object on the chart. But you can choose to add a number of the other fields with information to display, when you click on such an object. You have to select which data to include in your catalog version. For each catalog type, there is a minimal quantity of information that is required to draw the object on the chart. But you can choose to add a number of the other fields with information to display, when you click on such an object.
Line 50: Line 49:
   * **Save Project** allows you to create (if it didn't already exist) the "xxx.prj" file (xxx = catalog short name).   * **Save Project** allows you to create (if it didn't already exist) the "xxx.prj" file (xxx = catalog short name).
   * **Load project** restores the ".prj" file of your choice.   * **Load project** restores the ".prj" file of your choice.
 +
 +
  
  
Line 56: Line 57:
 ===== Page 1 ===== ===== Page 1 =====
 {{  en:documentation:catgen_1_dsl.jpg|}} {{  en:documentation:catgen_1_dsl.jpg|}}
 +
 In the first page you need to indicate the **Catalog type** in your catalog, this activates some specific options in the next pages. In the first page you need to indicate the **Catalog type** in your catalog, this activates some specific options in the next pages.
  
Line 64: Line 66:
 Click the **Open** button to select the catalog text file. The large catalogs are often split in many smaller files, in such a case select all the files at the same time, the file order doesn't matter. Click the **Open** button to select the catalog text file. The large catalogs are often split in many smaller files, in such a case select all the files at the same time, the file order doesn't matter.
  
-Choose **Output Catalog Type**. With SkyChart V3, you can choose text file output for small catalogs.+Choose **Output Catalog Type**. With SkyChart V3, you can choose text file for small catalogs. In this case no data is written out but just a file descriptor that let the program to access your original text file. This way a change to the file is reported to the chart without having to run Catgen again. 
 + 
 +If you select "Text file catalog" you also have the option to specify an "Update URL" that point to the latest version of the file on the Internet. This way you can get the last version by a single click in the catalog setup.
  
 Click the **Next** button to go to the second page. Click the **Next** button to go to the second page.
-\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ +\\ \\ \\  
  
  
Line 79: Line 83:
 ===== Page 2 ===== ===== Page 2 =====
 {{  en:documentation:catgen_2_ol.jpg|}} {{  en:documentation:catgen_2_ol.jpg|}}
 +
 Select the input format of the coordinates: Select the input format of the coordinates:
   * **RA options** Hour, minutes, seconds or decimal Hour or Degrees, minutes, seconds or decimal degrees.   * **RA options** Hour, minutes, seconds or decimal Hour or Degrees, minutes, seconds or decimal degrees.
Line 96: Line 101:
   * **Edit line operation** (see below) set character strings that will be recognized for drawing operations (comma separated values). Click return when ready.   * **Edit line operation** (see below) set character strings that will be recognized for drawing operations (comma separated values). Click return when ready.
   * **Edit color** (see below) set character strings that will be recognized for lines color (comma separated values). Click return when ready.   * **Edit color** (see below) set character strings that will be recognized for lines color (comma separated values). Click return when ready.
 +
 {{en:documentation:catgen_2a_ol.jpg|}} {{en:documentation:catgen_2b_ol.jpg|}} {{en:documentation:catgen_2a_ol.jpg|}} {{en:documentation:catgen_2b_ol.jpg|}}
  
 **Nebulae Options**  **Nebulae Options** 
 {{  en:documentation:catgen_2_dsl.jpg|}} {{  en:documentation:catgen_2_dsl.jpg|}}
 +
   * **Dimension and Units** default dimensions (in case of empty fields: no data) and unit.   * **Dimension and Units** default dimensions (in case of empty fields: no data) and unit.
   * **Logaritmic scale** check this box if the field used for size is a flux, i.e. for a radio source catalog.   * **Logaritmic scale** check this box if the field used for size is a flux, i.e. for a radio source catalog.
   * **Edit Object Type** set character strings that will be recognized to set the object type (comma separated values). Click return when ready.   * **Edit Object Type** set character strings that will be recognized to set the object type (comma separated values). Click return when ready.
   * **Edit Units** set character strings that will be recognized to set the unit size (comma separated values). Click return when ready.   * **Edit Units** set character strings that will be recognized to set the unit size (comma separated values). Click return when ready.
 +
 {{en:documentation:catgen_2a_dsl.jpg|}} {{en:documentation:catgen_2b_dsl.jpg|}} {{en:documentation:catgen_2a_dsl.jpg|}} {{en:documentation:catgen_2b_dsl.jpg|}}
  
 Click the **Next** button to go to the third page. Click the **Next** button to go to the third page.
 +
  
  
Line 156: Line 165:
 If you check **Use this field name as an Alternate name** the string value can be used as an alternate name for the object. This name is used in the case the main name (Catalog Id) is missing or it can be added to the index file of the catalog.\\  If you check **Use this field name as an Alternate name** the string value can be used as an alternate name for the object. This name is used in the case the main name (Catalog Id) is missing or it can be added to the index file of the catalog.\\ 
 {{  en:documentation:catgen_3_dsl_adv.jpg|}} {{  en:documentation:catgen_3_dsl_adv.jpg|}}
-**Advanced** button enables you to compute a linear transformation of the catalog data, it is active only for numeric values.\\  + 
-A use is to obtain the object coordinates when not directly available from the catalog.\\ +The **Advanced** button enables you to compute a linear transformation of the catalog data. It is active only for numeric values.\\  
 +A use can be to obtain the object coordinates when these aren't directly available from the catalog.\\ 
 If the catalog gives the object position in millimeters in the plate from the galaxy center (i.e. ra: 0h42m42.00s dec:+41°16'00.0") and you know the top of the plate is at North and the plate scale is 2 seconds per millimeter.\\  If the catalog gives the object position in millimeters in the plate from the galaxy center (i.e. ra: 0h42m42.00s dec:+41°16'00.0") and you know the top of the plate is at North and the plate scale is 2 seconds per millimeter.\\ 
 The approximate (without plate correction) coordinates in degrees are: The approximate (without plate correction) coordinates in degrees are:
Line 175: Line 185:
 ===== Page 4 ===== ===== Page 4 =====
 {{  en:documentation:catgen_4_dsl1.jpg|}} {{  en:documentation:catgen_4_dsl1.jpg|}}
 +
 In the last page you set the options for the generated catalog. In the last page you set the options for the generated catalog.
  
Line 189: Line 200:
  
 {{  en:documentation:catgen_prog_dsl.jpg?300|}} {{  en:documentation:catgen_prog_dsl.jpg?300|}}
 +
 A progress box shows you the current operation. You can interrupt the process by pressing the **Abort** button. In such a case the catalog will not be usable.\\  A progress box shows you the current operation. You can interrupt the process by pressing the **Abort** button. In such a case the catalog will not be usable.\\ 
 After the build is complete, the progress box is closed. You can exit the program with the **Close** button. After the build is complete, the progress box is closed. You can exit the program with the **Close** button.
Line 197: Line 209:
  
 If all worked well, you are now ready to use the catalog. Refer to **[[http://www.ap-i.net/skychart/en/documentation/catalog#catalog|SkyChart documentation]]** to activate the catalog. If all worked well, you are now ready to use the catalog. Refer to **[[http://www.ap-i.net/skychart/en/documentation/catalog#catalog|SkyChart documentation]]** to activate the catalog.
 +
 +===== Notes for Linux and Mac users =====
 +
 +Please read carefully if you plan to make a big catalog with 9537 files.
 +
 +As it read lines from the input files Catgen need to randomly write the data to
 +the 9537 output files, depending of the star coordinates. So the 9537 files need
 +to be open at the same time.
 +
 +As Linux and Mac OS are multi-user operating system there is safe-guard to avoid one user
 +to eat all of the system resource. For a standard Linux installation for example the
 +maximum number of file a user can open at the same time is limited to 1024.
 +
 +On Mac OS X the way to do may vary depending on your version. See the first answer [[http://unix.stackexchange.com/questions/108174/how-to-persist-ulimit-settings-in-osx-mavericks|to this question]].
 +
 +On Linux you first need to change the system limit in the file
 +limits.conf :
 +<code>
 +sudo vi /etc/security/limits.conf
 +</code>
 +add this two lines at the end of the file:
 +<file>
 +* soft nofile 1024
 +* hard nofile 32768
 +</file>
 +I recommend you keep the default value of 1024, this can be a good protection
 +again a runaway process.
 +
 +Save the file, and logout from the system as this file only apply at next login.
 +
 +Then open a terminal window and type : 
 +<code>
 +ulimit -S -n 20000
 +</code>
 +and from the same terminal type : 
 +<code>
 +skychart
 +</code>
 +This is very important as the limit change is only for the current instance of
 +the shell, not system width.
  
en/documentation/catgen.1231801039.txt.gz · Last modified: 2015/11/06 20:33 (external edit)