TNG Forum   |   TNG Wiki
« Help: Most Wanted   |   Help: Post-Import Utilities »

Help: Import / Export

GEDCOM Import   |   GEDCOM Export

GEDCOM Import

This page allows you to import all data from a standard GEDCOM file into a particular tree.

Before Importing: Use your desktop genealogy software to create a standard GEDCOM file (5.5 or 7.0 are preferred, but 4.0 will also work). You may choose to exclude private information for living individuals, but this is not necessary. It is also not necessary to exclude LDS information, as this can be filtered as well depending on user rights.

After you've created your GEDCOM file, the next step is to get it to your web site. Please take note of the following items on the page:

Import GEDCOM From your computer To upload and import your file to your site without using FTP, click on the "Browse" button and locate the file on your hard drive. The name and path of your file will appear in this field after you select it. NOTE: If your GEDCOM file is large (> 2Mb), you can try "zipping" it first (TNG will automatically extract the file after it is uploaded). If the resulting file is still large, you might need to check with your hosting provider before you can upload your file this way, as the server may have a maximum size limit for files uploaded via a web form. If you receive an error during the import, this is likely the case. Try using FTP to copy your file to the GEDCOM folder instead, then import it from there (see below).

OR Import GEDCOM From web site (in GEDCOM folder)

If you used FTP or an online file manager to transfer the GEDCOM or ZIP file to the GEDCOM folder on your site, enter the exact name of your uploaded file here, or click on the "Select" button to locate it on your site. It must be in the GEDCOM folder, or the Select button will not find it. NOTE: If you see a list of files but they are not from your GEDCOM folder, you likely have a path problem. Check your Root Path (Admin/Setup/General Settings) and your GEDCOM path (Admin/Setup/Import Settings).

Note: If you upload/import a ZIP file and include actual media in the file, TNG will extract the media to the appropriate media folders (eg, "photos" or a subfolder of "photos"). See Admin/Setup/Import Settings for more options related to embedded media.

Accept data for all new event and attribute types

Your GEDCOM file might contain events that TNG considers as non-standard events. Normally, new event or attribute types included in a GEDCOM file are entered into the database, but the data is set to be ignored. You would need to change the status of such a type to "Accept" in order for events or attributes of that type to be imported (in other words, you would need to import your file twice). If you check this option, TNG will automatically set all new event or attribute types to "Accept", and all your events will be imported the first time.

Import Event / Attribute Types Only (no data is added, replaced or appended)

Checking this option will cause only custom event or attribute types to be imported (see Admin/Event / Attribute Types). All other data is ignored. This is an ideal option to select during your initial setup, as it will allow you to see which custom events you have in your gedcom. You can then select which ones to accept and which ones to ignore before importing your entire database.

Destination Tree

Select a Tree to receive the imported data (required). If the tree to receive the data does not yet exist, click on the "Add New Tree" button to create it. A small popup box will appear and allow you to enter the information for the new tree.

Replace all current data

If you choose this option, all your previous GEDCOM data (people, families, children, sources, repositories, events, notes, associations and citations; not media or anything else) will be deleted prior to the import. NOTE: Links to media will be preserved as long as the people/family/source/repository IDs in your new GEDCOM match the IDs in your existing data. Most desktop genealogy programs assign permanent IDs to each person/family/source/repository, but a few do not. If you have linked in any media items, please check your new GEDCOM to make sure the IDs match before you import, no matter which of these import options you select. It might also be a good idea to backup your tables before performing an import (see Admin/Utilities to perform a backup).

Replace matching records only

With this option, new records are added and matching records are replaced (matches are determined by ID only). Old data is not deleted.

Do not replace any data

New records are added, but any matches are ignored (not replaced).

Append all records

All records are imported, regardless of existing data, but their IDs are recalculated. IDs numbers for the imported records will be added to the first available number (or a number you specify) to create the new ID numbers.

Uppercase all surnames

Check this box prior to importing if you want all incoming surnames to be converted to upper case. The surnames will be stored in the database this way, so the process is not reversable unless you import your GEDCOM file again.

Do not recalculate Living flag

If you chose "Replace matching records only" above, you will also see this option. Check this box prior to importing if you do not want the Living flag to be recalculated for individuals that were already in the database.

Replace only if newer

Matching records will only be replaced if the incoming record is newer than the one in the database. This is based on the "Last Modified" or CHAN date associated with the record in the GEDCOM.

Import media if present

If your GEDCOM contains links to media, selecting this option will allow TNG to import them and set up the appropriate links. You must still copy the physical files associated with these links to the appropriate folders on your site via other means (e.g., FTP). If you do not want to import any media links, make sure this box is not checked prior to performing the import.

Start IDs at first available number/Start IDs at

If you chose "Append all records" above, you will also see this option. Check the first option to have TNG add the numbers for the incoming IDs to the first available slot in each category (individuals, families, sources, repositories) to create the new IDs. Check the second option if you want TNG to create the new IDs by adding the incoming ID numbers to a specific number (same for each category). If you choose this option, be careful that no records exist in the indicated range or you will have collisions.

Old style import

Prior to TNG 7, the import did not show a progress bar. Instead, a counter was printed repeatedly to the screen for the number of people and families imported. The new number would always be displayed next to the old number, and soon they would be scrolling off the page. The progress bar is cleaner and more intuitive, but in some cases the old import actually works better because more information is being displayed on the screen, a little bit at a time. If the new import fails for you, you might try checking this option to see if the old import performs any better.

When you're ready, click on the Import Data button to begin the process. You should see a progress bar and a series of counter track the individuals, families, sources, notes and places imported (note: only places with associated latitude/longitude data are counted). A final message will indicate the import has finished properly.

"Resume" Feature

If you do not see the "Finished" message, your server may have killed your import process for running too long. If this happens to you, go to the Admin/Setup/Import Settings page and check the box next to Save Import State. Then return to the Import page and try your import again. If the same condition occurs, the import should now be able to restart itself.

Top

Changes made by the Gedcom Import Mediatype mod

If your source database has only one folder for file of all Mediatypes (i.e. "Collections" - Photos, Documents, Histories, etc.), then, in most cases, the Gedcom Import process is limited in how it can identify the Mediatype of new Media Items. In the native Gedcom Import, image files on such TNG sites will (almost) always be assigned to the "Photos" collection. And, after the Gedcom Import, an administrator will typically have to reassing some new image files to other collections. In fact, almost any collection can contain image files.

However, if (as is likely) the Photos collection is large, it will be difficult to find the new media items and reassign them. The same problem exists for PDF files, though they are typically assigned to the Histories or Documents collection by default. (The default collection for PDF files is not a TNG system parameter; it is determined by PHP code in the file tngfiletypes.php.)

The Gedcom Import Mediatype makes the post-Gedcom-Import media item reassignment process MUCH simpler. Two options:

  1. A comma-delimited list of filetypes (e.g. jpg,pdf), and
  2. A TNG media collection (frequently "Videos")
determine which media items get reassigned. Assuming the specified collection doesn't contain many media items other than the new ones, it should be easy to identify the new media items that belong in, say, Documents, Histories, or Headstones. Then, the image files that remain in the specified collection can be reassigned to "Photos and the remaining PDF files can be reassigned as appropriate.

Defining the new options

  • The default values (for a given TNG site) of the two options are specied as mod options for the Gedcom Import Mediatype mod.
  • The two options are also included in the Gedcom Import "kickoff form" so you can confirm or change them for any specific Gedcom Import.
The new options will be shown in the Gedcom Import kickoff form only if the "import media" option on that form is checked. When it is unchecked, any media items in the Gedcom file are ignored, so these options are irrelevant.

Top

Changes made by the Gedcom Import Purge mod

1. Purging Place Records

The Gedcom Import Purge changes the Gedcom Import process to limit the number of places that are purged. As the import starts, it no longer purges
  • Any Places on sites where there are multiple trees, and just one Place list (since such places might be used by trees other than the one being imported),
  • Places with a placelevel value (since placelevels cannot be obtained from a Gedcom file), or
  • Places with a Medialink (since Place medialinks can only be created from within TNG, not through Gedcom import).

2. Purging Medialinks

The native Gedcom Import Process purges all Person, Family, Source, Repository, Citation, and Event data from the tree being loaded, because all such data is expected to be supplied by the Gedcom file. There is simply no decent way to mix Gedcom import of these record types with TNG data entry (at least when you are importing an entire tree.)

But media information is different. It is very common for TNG administrators to import some media items and medialinks from Gedcom files, and to do manual uploading and data entry of some media items, their descriptions, and their links to other database objects. Since TNG does not natively provide a way to distinuish media items and medialinks that are created through Gedcom Imports from those that are created through TNG data entry, all a native TNG Gedcom import can do is to leave all media items and medialinks intact, causing Medialinks records that have been removed from the source database to remain in TNG after the import.

The Gedcom Import Purge mod addresses this problem by

  1. Installing a one-time setup program that creates a new database field in the Medialinks table that indicates whether a medialink was created during a Gedcom import or was defined "manually" - through TNG data entry programs. The new field is named "createdfromgedcom", and it is set to 1 for medialinks created during a Gedcom Import.
  2. Purging (at the beginning of a Gedcom Import) all medialinks that are flagged as being created by a Gedcom Import (but doing so only if the user checks the mods "Purge Medialinks" checkbox in the Gedcom Import kickoff form.
  3. Adding a form field in the Medialinks section of the Edit Media form so that users can view each medialink's "createdfromgedcom" flag. (The form allows the user to set or reset that flag, but the flag really represents how the medialink was created, so it would be fairly unusual to change its setting.)
For a period of time between incremental versions of TNGv12, the Gedcom Import would invalidate all existing Citation Medialinks and leave in place, and the invalid Medialinks would sometimes show up TNG pages such as Person Profiles. To respond to this problem, the Gedcom Import Purge mod added a "Suppress the creation of citation medialinks" button on the Gedcom Import kickoff form, and changed the Gedcom Import process to act on that button.

This problem has been resolved, so, really, there is virtually no reason to have the "Suppress the creation of citation medialinks" checkbox any more. About the only thing it accomplishes is to suppress linked media item thumbnails on source citations in the Person Profile.

See the Gedcom Import Purge Wiki article for more information.

Top

GEDCOM Export

This page allows you to export all data from a particular tree to a standard 5.5 GEDCOM file. The file will be available for download in your GEDCOM folder (as indicated in the Import Settings) and will be given the name of the tree's Tree ID plus ".ged".

Branch

Choose a branch from the list to include only people and families from this branch in the exported GEDCOM file.

Exclude Living/Exclude Private

Checking these boxes will completely exclude living or private individuals and families from the exported GEDCOM file.

Export media links

Check this option to include information pertaining to all photos, histories and other media linked to individuals, families, sources and repositories in the selected tree. This information includes the media file name, description and notes.

Zip and include media files

If you check the option to export media links, you also have the option to zip the finished file and include the physical media files mentioned in the GEDCOM file by checking this box. Take note that this could make your file extremely large.

Local paths for Photos/Documents/Headstones/Histories/Recordings/Videos

To prepend a common path to each media file name (ie, "C:\myphotos\" or "..\genealogy\"), check the "Export media links" box, then enter that path in the appropriate field (be sure to include a trailing slash). If these lines are left blank, only the file name and path stored in TNG for each media item will be exported in the GEDCOM's "FILE" tag.

"Resume" Feature

If you do not see the "Finished" message, your data was not completely exported. If you do not see any numbers, or if you see numbers but do not see a "Finished" message, your server may have killed your export process for running too long. If this happens to you, go to the Admin/Setup/Import Settings page and check the box next to Save Import State. Then return here and try your export again. If the export fails to run to completion, you will now be able to click a "resume" link at the top of the page to resume the export.

Totals

When the export finishes, you should see a summary of how many people, families, sources, notes, repositories, media and places were exported. It should be noted that only places that have latitude/longitude information or notes will be tallied in the place total.