or
Examples of aggregation requests: count of linked life events
{$heurist->getRecordsAggr(array('id','count'), '{"t":"48","linkedto":"[ID]"}', $r) }
average height of persons linked to given record $r
{$heurist->getRecordsAggr(array(1014,'avg'), '{"t":"10","linkedto":"[ID]"}', $r) }
It is possible to execute several aggregation requests per call. The first parameter of getRecordsAggr is an array of pairs using either sum, count or average: field id: sum | count | avg
It is also possible to perform defined query and work with the result set as usual in smarty:
{$records = $heurist->getRecords('{"t":"48"}')}
{foreach $records as $r} {* Start records loop, do not remove *}
{$rec = $heurist->getRecord($r)}
{$rec.recTitle}
{/foreach}
An optional second parameter for $heurist->getRecords can be the record ID
$heurist->getRecords('{"t":"12","linkedto":"[ID]"}', $r)
Hints and examples of good structure
Proposal @todo: need examples @todo: Keep fields separate EG family and given names
Clarity and Simplicity
A well-designed structure should be clear and simple. Avoid overloading forms with too many fields. Use tabs and collapsible blocks to organise fields in a logical and intuitive way. Each field should have a specific purpose and contain only relevant information.
Strategic Use of Hidden Fields
Hidden fields are useful for storing additional information that may be needed later without cluttering the user interface. For example, you can hide fields that are filled automatically or used for background processes.
Reuse of Base Fields
When creating fields, consider using base fields to reuse them in other record types. This ensures consistency and avoids data duplication.
Organise Linked Data
For record pointer fields, ensure that the relationships between records are well-defined. This guarantees that information is easily accessible and remains consistent when updated or modified.
Use Section Headings and Dividers
Use section headings to divide forms into logical parts. This helps users navigate long and complex forms. You can also use horizontal dividers to visually mark sections and make the structure clearer.
Use Default Values and Smart Increments
For fields like serial numbers, use default values or increments to simplify data entry and ensure consistency in values.
Controlled Data
Use term lists for fields where only certain values are allowed. This prevents entry errors and ensures data consistency.
Practical Examples:
For a person record, you could have fields like "Name", "First Name", "Type", and "Home Country".
An event record might include fields like "Title", "Start Date", "Location", and "Participants", where "Participants" could be a multivalue field with record pointers to people.
Document your database
Don’t cop out on writing proper help texts and descriptions of record types, fields and terms, even if you are the only person using the database.
A few extra minutes spent writing an explanation of the content of each field will ensure that the database is still interpretable way into the future, by you or by others if deposited in an archive (your descriptions automatically become part of the archive package). Do it as you are setting up the database, because you will never come back to do it later …
Download Structure
You can download the structure of your database in XML or plain text format using the "Structure (XML)" or the "Structure (Text)" links under the "Download" section in the Design tab.
Vocabulary and Terms
One of the most important features of Heurist is its ability to describe categorisation and provide structured terminology for data. Vocabularies allow users to define and standardise terms used across different records, ensuring consistency and accuracy in data entry and retrieval. By creating term lists and defining relationships between terms, users can easily classify and manage data according to predefined categories. This feature is particularly useful when dealing with large datasets, as it helps prevent errors, reduces ambiguity, and ensures that data can be analysed and compared meaningfully across different record types.
A term list is the set of predefined 'enumerated' values ('terms') that can be used within a particularly drop-down (i.e. a term list field type). Term lists are based on all or some items in a 'vocabulary'.
A **vocabulary **(parent term) is the underlying top level category of related 'child' terms (e.g. 'Language' is a vocabulary, while 'French' is a child term). Vocabularies can be nested (i.e. any child term can in turn become a vocabulary). The lowest level values are the terms. A term list can comprise a hierarchy of nested terms (i.e. nested), with any 'leaf term' being potentially a new term list. Term lists may be used for any form of classification or categorisation of preconfigured data, such as raw material, condition, period, religious affiliation, language, country etc. For example, a Language dropdown might have the following structure:
Language (this is the name of the field or term list)
English (term)
French (term)
Italian (term)
Spanish (term)
Etc.
When defining your database structure, you can create a new field based on the Terms List data type. You can then select what terms are to appear on the list, from the set of available vocabularies (Heurist provides a set of default vocabularies which you can edit and add to as required) and preview your choice. If a term is missing from a list you can quickly add it. The heading for the term list dropdown is the field name you have chosen for this field type.
Term lists are also used for specifying sets of relationships for Relationship Markers. For example, Family (Is Parent OF, Is Child Of etc.).
Selecting a Vocabulary and Terms
You can select the required term from the Terms List in the Vocabulary dropdown within the field. If a particular term does not exist in your chosen vocabulary, you can add a new term by clicking the gear icon. The Manage term window will open, allowing you to edit (pencil icon) existing terms or create a new one (+ ADD icon). The same applies to vocabularies.
The following options are available:
Add Terms. Use this to add a term to the current vocabulary (this does not add it to the base Vocabulary, just this instance).
Edit Terms Tree. Use this to edit the base vocabulary. (See Terms.)
Add Vocabulary. Use this to create a new vocabulary. (See Terms.) The new term is appended to the end of the term list (this also updates the base vocabulary).
The Manage Terms Screen
You manage the vocabularies that underlie term lists via the Manage Terms screen. Here you can edit the standard vocabularies, or create new vocabularies and terms. Click a vocabulary to show its available terms. Terms Pane
Actions for a vocabulary or term are available in the title bar of this pane:
Add a term
Import terms
Export terms
Find terms in all vocabulary groups
Editing a vocabulary or term
Select the vocabulary or term from the vocabulary hierarchy and edit its properties as appropriate.
Finding a term
Before adding a new vocabulary or child term to the vocabulary hierarchy, check if it already exists, by entering all or part of its name into the Find field to show all matching terms. If it does exist, it will appear in the box. You can click on the entry to highlight the term.
Creating a new, top-level vocabulary
If the new term does not already exist (see above), complete its properties as follows and click Add Vocabulary:
Term. The label for the vocabulary in the hierarchy.
Description. A user friendly description.
Standard Code. For standard codes such as three letter country indicators.
URL. For example, pointing to a semantic definition.
Image. You can use this field to attach an image (ideally 400x400 pixels) to a term (these will then show as a visual description next to the term on data entry screen). New terms must be saved first.
Status. You can set the status for any term (e.g. setting the Status to Approved prevents any additional changes to a term).
The term (vocabulary) is added (alphabetically) to the vocabulary hierarchy.
Adding a child (root) term
Check if the term you wish to add already exists (see above).
Select or hover over the vocabulary (or term if you are creating a hierarchy of vocabularies) you wish to add to and click Add Child (or click the Add Child button in the Properties dialog).
The new term is temporarily added to the hierarchy with the default name 'new term'.
Change the default name 'new term' to the name of the term you are adding and complete the other properties as appropriate (see above).
Click Save Term. New terms are added alphabetically but can be repositioned.
Repeat this process for each new term you wish to add (ensure you select the correct vocabulary).
Note. Adding a term to a vocabulary does not add them to the individual term lists for different fields, since these are individually selected from the complete vocabulary. You need to update the lists for each field to which these terms should be added.
Moving and deleting terms
To reposition a term/vocabulary, go to the Terms pane, then simply drag and drop it in the hierarchy (child terms are automatically included in the move):
To merge a vocabulary into another (i.e. combine their child terms), go to the **Vocabularies pane** drag and drop it onto the vocabulary you wish to keep:
To delete a term (or vocabulary), select it and click Delete.
Important. If you delete a vocabulary, all of its child vocabularies and terms are also deleted, and cannot be restored.
However it is not possible to delete any vocabulary or term which has been used in a record in the database.
Importing/Exporting a Vocabulary
Import
To import a vocabulary, select the vocabulary (or child term) and click the Import button
Step 1, prepare data for import as a comma or tab-separated file.
Paste the data or upload an existing CSV file (e.g. a previously exported vocabulary).
Step 2, define the parse parameters. Click Analyse, and preview the data to be imported in the lower pane.
Step 3, map columns to term field. When ready to import, click Import.
Export
To export a vocabulary, select it and click the Export button
The vocabulary is exported as a CSV file.
TUTORIAL
Accessing the ‘Vocabularies’ menu
The aim of this tutorial is to modify the structure of the database so that we can record, for example, the ideological affiliations of each world leader in the database. To do this, we need to create a Vocabulary of different political ideologies that our world leaders might espouse.
You can view, add and edit all the Vocabularies in your database by accessing the ‘Vocabularies’ menu in the ‘Design’ pane. Add a new Vocabulary called ‘Political Ideologies’:
Adding Terms to a Vocabulary
Once you have created a new vocabulary, you can select it in the Vocabularies menu, and then start adding terms to it. Add some terms such as ‘Communism’ or ‘Neoliberalism’ to your new ‘Political Ideologies vocab’:
Creating a relationship Vocabulary
Relationships are defined by Vocabularies. To create a Vocabulary for a relationship, the process is exactly the same as for creating a basic Vocabulary. However, you must ensure to tick the box ‘Use for relations’ when you create your new vocabulary. Add a new relationship vocab called ‘Poltical Offices’:
And you should also use a different naming convention. It is best to use verb phrases such as ‘is Prime Minister of’ for relationships, rather than simple nouns such as ‘Prime Minister’ (e.g. ‘Angela Merkel’ → ‘is Kanzler of’ → Germany). Add a few terms to your Political Offices vocabulary such as ‘is Prime Minister of’ and ‘is Dictator of’:
Hierarchical terms
Question : est-ce qu’on peut importer un thésaurus ou un vocabulaire controlé existant (ex. Opentheso) ?
À priori OUI à partir du moment où l’export peut se faire en csv ou SKOS (?) -> @todo lien vers opentheso Lookup
Heurist recognises a period-separated term suh as Stone.Igneous.Granite and Stone.Sedimentary.Limestone as creating a three level hierarchy (one can also specify that the periods can be treated as just periods within the labels).
Vocabulary terms can be structured as a hierarchy by dragging under any other term, thus creating a tree structure.
A simple example of such a structure (trees are not limited to two levels):
Clicking on merge into target term allows terms to be combined when dragged onto anotehr term - any term which is merged with another will be replaced in any records that use it with the result of the merge. A verification popup is displayed before any merge as the result is irreversible.
Reordering terms in the vocabulary tree
The
icon which appears on hover over a term will pop up a window which allows terms in that branch of the vocabulary to be reordered by drag and drop
Effects on search
Controlled tree vocabularies are useful for drop down choices as they allow search on a general term (eg. all Bénédictins) or on a more specific term (eg. Ordre de Fontevraud)
###To be continued !!! 04/03/2025
Editing terms
Term descriptors:
standard value, label, semantic you are i, image
flag terms
Setting Inverse Terms
By default, terms are 'non-directional'; that is, the same relationship term is used whichever the direction of the relationship (e.g. Painting > Linked to < Artist). However, if the relationship term does have an inverse that needs to be described (e.g. Versions > IsEditionOf has the inverse Versions > HasEdition), you can add it. Note. An inverse term must already exist in the Vocabulary tree list; if not, create it first.
Link terms
change field type
CSV export and import to modify field content
Record ownership
Add record URL with tags and field values and ownership
Sustainability - no plugins
@todo This does not belong here
Many systems provide basic functionality in the core product and depend on plugins for for quite common functions such as field formatting for data entry or export of common file types. This is a recipe for disaster, for example Drupal has more than 40,000 modules many of which only apply to specific versions, have not been completed to usable condition or no longer work.
=== Define Connections ===
Connect Data
A powerful feature of Heurist is the ability to relate or link records together to connect your data in a meaningful way without the complexity one might be familiar with in relational systems. Relationships are immediately available between ANY record types, without any further work, but these can also be constrained through a constraints menu so that only particular relationship types are allowed between particular pairs of record types, and this also allows constraint of the number of relationships allowed (e.g. one can only have two parents or a specimen bag can only belong to one context). However we also have two powerful methods for embedding connections directly in the records so that they appear contextualised in the data entry form: Record Pointers & Relationship Markers. For step-by-step instructions to create new pointer fields or relationship entities, click the links below:
Below, we explain the theory behind pointers and relationships. Which should you use, when and why?
Making connections
Record pointers
'dropdown' presentation with search
adding new values automatically linked to record
Child record pointers
Only one parent
Invisible back pointer, but it is in fact accessible as
cannot/should not use as stand-alone
'knows' its parent so cn make use of fields in parent
In which direction should you make connections???
Use of the visualisation diagram to make connections
Relationship markers
Relationship terms
Relationship vocabularies
Inverse terms / reflexivity
Pros and cons of using relationship markers
Mediaeval monks as an example
Intermediate records
Referencing bibliography types
Sub-records
Use for varying attribute sets
Use to describe elements eg. of a tool or a structure
Sub-record creation function
Use with caution, non-reversible
Record Pointers
The simplest way to connect two records to one another is using a Record Pointer field. In most cases, a record pointer will be sufficient. For example, if you wish to record that a particular Building is situated in a particular Place, it would usually be sufficient to have a field called 'location' in the Building record which simply points to the 'Place' where it is located. However, Heurist provides many additional ways of linking records to one another, when the simple Record Pointer solution is inconvenient.
A record pointer is a field within a record that defines or references a one-way link to one or more specific record types. You define pointers between data when you create the database structure. The type of a pointer can be constrained so you can only select a record of a particular type (or types).
Similar to term lists, pointers allow a field to be populated from a controlled list, but in this case the list is all records in the database of a particular type, or types. This effectively ‘embeds’ all the information from the chosen record in your current record (but it is only stored once, however often it is ‘embedded’).
Typically, record pointers are used when there is a specific known relationship. For example, to identify people (authors, owners, actors, ..), multimedia items (pdf, images, video), events, places, organisations etc. with specific relationships to a record. Record pointers can define relationships between heterogeneous records (e.g. event with person, building with date etc.).
For example, imagine a record about a chapter in an edited book. It has one or more authors and it belongs to a book. But it may share the author(s) with many other books, book chapters, articles and so forth, and the book with a dozen or so other chapters, each with different authors. Rather than entering the author(s) as text fields and repeating this information for every chapter in the book, you can create records for each of these Author entities and link them into the record for the chapter. You can then use the Author(s) field (which in this cases is a repeatable field) to select exciting authors in the database, or if they do not already exist, create them. In the same way you can create book records, series records, publisher records and so forth, and simply point to these records instead of re-entering the data.
Using pointers saves typing, reduces data entry errors, and ensures a continuous connection between records that share the same source material (e.g. authors, books, publishers etc.).
Use record pointer and relationship marker fields
Record pointers are the workhorse for quickly and easily building simple relationships between records. Use a record pointer field to establish a hierarchical relationship through a pointer to a parent record (e.g. a chapter belonging to a book or a photograph belonging to a collection) or to indicate records with a specific role in relation to the entity being described (e.g. the author of a book, the producer of a film, the venue where a play is performed, a birth or commemoration event, a qualification).
Relationship markers are similar to pointers but carry additional information; minimally, a relationship type, but also commonly a date range over which the relationship is applicable. Relationships are useful where there are lots of potential types of relationship (e.g. roles that people may play in relation to a theatre production), as an alternative to defining a separate pointer field for each role.
They are also useful where the relationship has a limited duration (e.g. relationships of employment, patronage, residence or exhibition/loan).
As a general rule, use a pointer field, constrained to a specific record type (e.g. place, person, series, component), where you will record a single value (e.g. parent) or a small number of values (e.g. authors) which have an unequivocal relationship with the entity being described and where multiple pointers are all equivalent (although they may be ordered—authors being a good example).
Use a relationship marker field where you do not know a priori which relationships will be present and/or there are numerous possible relationship types, or the relationships have a temporal range, or the relationships are subject to interpretation and you need to provide supporting information through notes or references.
Relationship markers
Relationship marker fields provide a built-in method for connecting entities with typed relationships and dating. This is very useful for things like relationships between people eg. family or associates, or between people and groups eg. organisations, associations. It is valuable because Heurist can automatically 'reflect' the relationship so that a relationship marker placed in both records will show the relationship from the perspective of the record where it is located. So if A is shown as Master Of B in A's record, B will be shown as Student of A in B's record (see example of Briuno of Cologner and Willigis, Archbishop of Mainz below).
Relationship records can include start and end date of the relationship as well as other attributes such as notes and bibliographic references, and the user can add additional attributes if they wish, as with any other record type
However, although relationship markers can define the set of relationships which are allowed and the types of entity which are to be related - so we can have family connections of people and stratigraphic relations of archaeological contexts in the same database - relationship records are limited by the fact that there is only one type of Relationship record. So, if one adds additional attributes they will be added to all Relationships.
This may not make a lot of sense if one starts to add, for instance, fields for stratigraphic drawings, photos and field notes describing the stratigraphic relationships, which will then appear for family relationships; although its is perfectly OK just to ignore them, it's inelegant. This is where intermediate records connecting entities come in to play. They take on much the same role as relationship records, but each type of relationship will have its own record type with the attributes specific to that relationship.
However it may be worth creating an intermediate record even if there is only one type of relationship in order not to overload relationship records with additional fields and with a view to adding other intermediate record types in future.
Relationship markers A relationship marker is a record that defines a two-way link between two records that you wish to connect. Relationship markers allow connections to be established between any two types of entity, but also allow the type of connection to be recorded via a separate relationship record. What the relationship marker does is build in the relationship to the databases structure and prompts the user as they build their database. It provides structure as to what relationships the user can build.
**Note. **Relationship pointers differ from relationship markers in that they create a direct one-to-one link between records, without an intermediate relationship record, which in some instances may be the preferable solution.
(See ‘When to use a pointer and when a relationship?’ below).
The relationship marker is implemented as a separate record that links two records together, regardless of type. All relationship details are stored in the relationship record itself, which has two fields that point to the source record and the target record of the relationship. The relationship marker field is embedded directly in the data entry form – it does not actually contain any data itself, instead it acts as a marker (or prompt) to the user to create a new relationship record (‘show this type of relationship at this point in the form’).
Relationship markers may be further constrained to specific record types and a limited set of relationship types appropriate to that point in the form; the constraints restrict the term list (of relationships available) and the target record types. Relationship markers are useful in recording connections that are less standardised. For instance, have lots of different options (such as stratigraphic relationships or family relationships by birth) or have a time-limited component (such as museum loans or personal relationships by marriage or association) or otherwise require additional information (such as assignments of connections which require interpretation and explanation).
A good example of a relationship is that between a brother and sister. You can use a relatinship to represent Jack being Jill's brother, as in the diagram. In this case, there are three entities at play: Person(Jack) + Relationship(Siblinghood) + Person(Jill). In this case, Heurist automatically deals with Jack and Jill's genders, and implies that Jill is Jack's sister as soon as you enter that Jack is Jill's brother.
Child Record Pointers
A somewhat similar usage is to break down sets of attributes which apply to components of an entity or only appy to particular subtypes of an entity. For example:
Components: individual scenes (components) of a painting or decoration panel might be recorded as sub-records, and they might have further sub-records describing individual figures or motifs.
An architectural structure might similarly be broken down into components, some of which may be repeated eg. rooms, doorways, while others may only occur once eg. roof, with each of these component types having its own distinct set of attributes.
Subtypes: A stone artefact might have general attributes such as type of material, weight, dimensions, measurements and artefact class eg. ground axe, core, scraper etc., and then for each of these classes a sub-record, of which only one will be present, describing the more specific attributes relating to that class of object.
In the Intermediate records section, we used the example of a database about plays and the theatres where they were performed. In that example, we suggested that you might create a 'Production' record type to link plays to theatres. Now each 'Production' would usually, by definition, be a 'Production' of only one particular play – this would be a good use case for a Child Record Pointer. By making the 'Production' a child record of the 'Play' record type, you ensure that each Production is linked to a play and to only one play. This also makes it easier to read your database, as the 'Productions' will be neatly listed in the data entry form for each Play, and the Play for each Production will appear prominently at the top of the data entry form for each Production. In all of these examples, the sub-records are records in their own right, and can be seen as such in the database, but they 'belong to' a specific parent record. This is implemented by making the pointer from the parent record to the component or subtype record a child record pointer (the lefthand image shows only the most relevant fields from the record pointer field modification form, the righthand image shows how the child record indicates its parent):
Sub records and child records / Intermediate records
Relationship markers are ideal when you wish to record many different types of relatively simple relationships between different records. For example, Relationship Markers are ideally for recording family relationships – there are many different types of family relationship, but from a data perspective most family relationships are quite simple (some simply is someone else's mother). However, if you want to record a more complex interconnection between two records, then you may need an intermediate record type. For example, imagine that you wish to record where a particular play was performed. You have a number of plays in your database, and a number of theatres. Now a play is not simply performed in a theatre – each production potentially has a different cast and crew, runs for a different number of weeks, uses a particular text or version of the play and so on. So instead of creating a Record Pointer or Relationship Marker that directly connects a play to all the theatres where it was performed, you may wish to create an intermediate record type, such as 'Production', which sits in between plays and theatres. The 'Production' would record which play was produced and in which theatre(s). You could also record any other information you liked about each Production, such as the cast and crew, acting style, budget and so on. Sometimes you may need to create a number of intermediate record types to link two records together. Thinking about linking records in this way can be a good way of building the structure of your database.
Deletion of child record pointers
Move fields into sub-records
This is a complex funciton. We recommend, therefore, making a backup copy of the database with Admin > Clone before running it (and perhaps trying it on a clone before running it on your production database).
Using data in intermediate and sub-records/child records
Having recorded data in records which are connected to the records of interest, either one step or two steps removed, how do I access the fields in these records, for instance to find all the books illustrated by a particular illustrator, the images containing a particular type of motif, all the buildings with peristyles longer than 10 metres or the silcrete artefacts with polish on the lefthand edge? We can access these fields in several places: Constructed title Filter builder Facets builder Custom reports
Pointer or Relationship? The primary difference (as shown in the diagram opposite) between a record pointer and a relationship marker is that in the first instance the relationship details are stored in the record whereas in the second instance the relationship details are stored in the intermediate relationship record, which gives you more control over the relationship (e.g. specifying the type of relationship, date range, label, annotations etc.). The simple rule is, if you simply need to identify a fixed type of relationship, such as an incontrovertible whole-part or a specific function such as Excavator, use a Pointer field. If you want greater richness, such as specifying an open-ended list of roles, e.g. for a film Director, Producer, Gaffer, Actor, Cinematographer, etc. and to enrich those roles with temporal limits, annotation and so forth, then use a Relationship Marker field. When to use record pointers
Aim to use a record pointer if possible, that is where the relationship between two records is not time-limited. Record pointers can also be set to parent-child (whole-part) where there is such a relationship.
If you have several types of relationship you can use several record pointers.
However, if there is a long list and/or time limits eg. in family relationships, or where you wish to add notes and referencing information to explain each relationship individually:
Either: use a relationship marker field (which allows a list of relationship types in the relationship type field, plus space for additional fields such as time, notes and referencing information);
Or: use a record pointer field to a new record type (intermediate record) which expresses the relationship and any other information you wish to record. In this case you are effectively introducing a typ of relationmship record, but one which does not use the special functions (notably inverse relationship types) of relationship records.
Where some entity (e.g. an author), is referenced by many records. The data about that entity (name, title, date of birth, location, roles etc.) can be entered once into the record describing the entity and then referenced from as many other records as you wish. This is preferable to listing all the related records in the single record to which they are related (which is why we reference an Author for each book or artiucle they wrote, rather than listing al lthe books and articles under each author).
For resource pointer fields, where you wish to constrain the pointers to one or more specific record types. This is useful, for example, if you want a pointer to a person or organisation (e.g. as the owner of copyright) and want to make sure that this pointer can only point to one of these entities and not to, say, a website or an artefact. When to use relationship markers
If the relationship is not permanent (i.e. it has a time range, such as a person as emperor of an empire)
There are several different types of relationship possible between any pair of entity types (for example, an organisation can be related to people as director(s), owners(s), member(s), student(s) etc. Rather than creating separate pointer fields for each of these relationships, they can be created as relationship records with a range of relationship types)
The relationship is not unequivocal or has rich information associated with it, and therefore requires commentary, justification or bibliographic references (which can be entered as Interpretations or notes in the relationship record – there is nowhere to store additional information in a pointer field);
The set of relationships is open-ended or requires complex constraints, such as genealogical relationships which might be extended with new relationships, and where one might wish to specify, for example, that a person can have no more than four grandparents, only two of whom can be grandfathers.
By using relationships, you can record additional information about the relationship, including the type of relationship (from a list of allowable types), the date range of the relationship and notes about the relationship
Constructed titles and Title masks
One of the most powerful and underutilised features of Heurist is hidden-in-plain-sight. It is the title used to represent records in the results list, in connections, in reports and many other places.
The Constructed title is like the reference you might find in the bibliography at the end of a book: it uses a concatenation of important fields, sometimes shortened, to uniquely identify and summarise the database record in question. The constructed title is generated on-the-fly when the record is created or modified.
Constructed titles are used to represent records when they are listed in search results and as the visible representation of the record referenced in a pointer field (first image below) or relationship marker field (second image below).
Constructed titles can also be used in reports and visualisations, for sorting, in other constructed titles, and as the constructed title of connected records. We strongly recommend putting a little thought into the design of the constructed titles, as well-designed constructed titles can greatly improve the clarity and ease of use of the database.
The Title Mask defines the choice of fields in the constructed title. Title masks are one of the most useful, and perhaps misunderstood or under-used features of Heurist. A separate title mask, with different fields, is defined for each record type. The constructed value is used as the extended title displayed in search results and other lists.
The title mask builds a constructed title from the values of fields in the record.
Fields are identified by [ ] e.g. **[Title], pp. [Start_Page]-[End_Page] **might generate: "Alice in Wonderland, pp. 37-39" Conditional text
Add optional text before a field (if it has a value) or a different set of text if a value is not available by adding {\Text for existing value \Text for missing value} after a field, for example: [Starting_date] {\Starting date: \Start date unknown} will either generate: "Starting date: 04-11-1974" if there is a date, or "Start date unknown" if Starting_date is empty. You can also leave the value blank, in which case nothing will be output in the case of a missing value.
Inserting a literal square-bracket : use two consecutive square-brackets ([[ or ]]).
Inserting fields from the tree : The element names in square brackets should match field names for the record type, and this is ensured by providing a tree of available fields which can be inserted
Constructed titles can use fields in the parent record (connected by a parent-child record pointer), as we can see in this example:
Constructed title aka Record title or RecTitle
Setting the constructed title
To set the constructed title for a record type, edit any record of that type (or simply add a new blank record) and click on the Constructed title link left of the title at the top of the edit form:
which will bring up a dialogue allowing you to select the fields which you want to use to construct the title for every record of that type.
**HTML tags in Constructed Titles.
Admin > Rebuild Record Titles
This option recalculates all the constructed (composite) record titles, compares them with the existing title and updates the title where the title has changed (generally due to changes in the title mask for the record type). At the end of the process it will display a list of records for which the titles were changed and a list of records for which the new title would be blank (an error condition). Note. To check the validity of title masks, see Administration | Verify Title Masks. Result Title fields are scanned and title usage updated where applicable. The scan shows a list of records for which the titles were changed. This includes: number processed number marked for update number left as is (these are left blank due to incorrect formatting etc. and need to be checked manually via the next step) To view all updated records in the Search Results Pane (in a new browser window), click the view updated records link. Note. If the title is blank, update the record appropriately (see Define New Record Type | Title Masks).
Content to be merged or eliminated
show lots of tips and tricks of how to use them, notably when dealing with hierarchical entities Title masks allow you to define composite titles that can be constructed dynamically from field values.
**You can add simple html tags in the constructed title eg. for bold or link to put link to open an image referenced in the record purely by its name. Please remember to close tags. Bold, italic, underline, strong, emphasis and superscript are allowed, Others are stripped out automatically. If you need others, contact the Heurist team. Note. To verify title masks, see Masks provide the ability to build a composite title based on information taken from other fields in the record, on the fly. The title mask is a string into which field values are inserted to create an extended title for the record. The constructed value is used as the extended title displayed in search results and other lists. Fields in the record are indicated by square brackets. The element names in square brackets should match field names for this record type. For example, a Person record might have the fields: Given Name(s), Family Name, Title. In this case you could create the following title mask: [Family Name], [Given Name(s)] ([Title] A person whose Family Name = 'Smith', Given Name(s) = 'John', Title = 'Dr' will be rendered in the Title field as: Smith, John (Dr) Other people will be rendered appropriately. Fields in records that are referenced by the record through pointers can also be used. For example: [personpointer].[Last name] This pulls out a person's name from a person record pointed to by the current record. Additional text or punctuation can also be included. For example: [Title], pp. [Start_Page]-[End_Page] This renders the Title field and Start and End Page fields as, for example: Alice in Wonderland, pp. 37-39 To insert a literal square-bracket, use two consecutive square-brackets ([[ or ]]). Fields in records referenced by the record through pointers can also be used: [personpointer].[Last name] This gets a person's name from a Person record pointed to by the current record. To create a title mask
Once you have saved your record type, select the Edit Mask button. Note. To later edit the Record Type page, navigate to the Record Type page (go to Database | Manage Structure, select the relevant group and click the Edit icon for the record type.) The Record Type Title Mask Edit dialog displays:
Note. You can enter a mask directly into the field if you wish, or build the mask as follows.
Position the cursor in the Build Mask field.
Select the field(s) you wish to insert from the left hand column (this shows all available field markers in the current record, plus fields in records pointed to by this record) and click Insert Fields. You can repeat this step for each field or set of fields.
To add additional text around the field markers, enter the text without square brackets in the appropriate location.
When ready, you can test the mask using actual data. From the Test Mask dropdown, select any record, then click Test to view the result:
When the mask is correct, click Save Mask to save it. The mask will now appear in the Mask field.
Ch 06 : Populating the database (import, lookup & synchronisation)
Ch 06: Populating the database
1 Populate menu
1.1 Introduction
The [Populate] menu gets data into the database. You can create individual records via a form, upload data files such as a CSV file from a spreadsheet or an XML file from another database, synchronise with the Zotero bibliographic system, or upload and index media such as a collection of images.
1.2 Populate menu functions
Functions for adding and importing data.
New record - opens a data entry form to enter data for a new record
Upload Files
Delimited text / CSV - CSV upload wizard, splits complex CSVs into component record types
Zotero bibliography sync. - synchronise records with one or more Zotero databases
Heurist XML/JSON - import XML or JSON exported from another database, Heurist or other
Download template (XML) -- get a template for formatting data for import by the above
KML -- import spatial data, creating a new record for each spatial object
Media Files
Upload media files/images -- uploads individual files or directories, maintaining structure
Upload media from URLs -- uploads a set of files specified by URLs
Index external transfers -- scans media folders and add missed to Media Files, indexing uploaded files or external transfers
Create media records -- creates, updates and reads XML manifest files in the folders ; creates Digital Media records for all files uploaded to the database
IIIF Images -- upload IIIf images or manifests
Process IIIF manifests -- reads an uploaded IIIF JSon manifest and creates Canvas and Annotation records
2 Manual input
2.1 New record
This function creates a new empty record, ready for data entry. This is the primary means by which a database is populated manually by the users. By default and in order to speed manual data entry, the type of record created will be the same as the most recently created record. This default record type appears in italic; in this example, the default type is Person.
Clicking [New] button directly creates a record of the default type. A popup appears in which you can immediately begin entering data.
Below [New], hovering over [Settings] opens the slide tray showing the available record types in the database. To create a new record of a particular type, simply click on that record type.
Inside the record editing window, fields in bold red type are mandatory fields, which must be filled in order for the new record to be saved. There are range of options for editing both the record and also change the structure of the record (Modify Structure in the top left corner). It is not recommended to modify the structure of records unless you are an experienced user and have a good reason for doing so. In the bottom banner, there are several options for saving the current new record and then taking other actions:
duplicating the record (Dupe),
creating a fresh new record (New),
save the current record but remain editing it (Save),
save the current record and close it (Save + Close),
and close the current record without saving it (Drop Changes).
2.2 Permission settings
You can control and change permissions settings to all the data entry of a specific record type by clicking on [Permission settings] at the top of the list (right-hand panel below) which pops up on rollover of [New], or by clicking on [Settings] below [New] button. It offers additional controls over the new record parameters:
By default, records in a new database will be visible only to logged in users. [Settings / Permission settings] brings up a dialogue allowing you to control the type and permission settings for future additions (cf. chapter 2 Roles and permission).
This can be used not only to determine the future record type and permissions which will be created when you click on [New], but also provides a URL which can be bookmarked or added to a web page to create new records with those specific permissions. The use of a tag or tags can be used to flag new records added, for example, by guests, that can be retrieved for editorial vetting. Other values can also be set with suitable parameters in the URL.
3. Upload Files
3.1 Delimited text / CSV
3.1.1 Presentation
Delimited text / CSV upload is the primary means for populating your database with bulk data. This tool is used to parse delimited text, comma-separated or tab-separated variable (CSV or TSV) data, and then organise that data into structures that are compatible with Heurist. The import tool is a very powerful way to populate your database, but it can be a complex process. It is important that the data is as clean as possible, prior to import. If you are unsure about any step in the import process, please consult the Heurist Help System, watch the walkthrough video, or contact Heurist community mailing list. There are three ways to begin using the Delimited texte/CSV upload workflow:
Upload new file -- an existing CSV or TSV file from your desktop
Select previously uploaded file -- these files appear in a dropdown menu
Paste delimited data below -- you can paste data directly from the clipboard. Please observe the conventions for representing data, including using column labels in the first line, proper line terminations, quotes, and special symbols. These are explained in the help sidebar, and also in a dedicated page in the Help System.
3.1.2 What is CSV?
CSV, which stands for 'comma seperated values', is a simple text-based format for saving spreadsheets or tables. Data is stored as text. Each line in the text file represents a row of data, and commas are used to seperate each column (hence 'comma-seperated'). Consider the below example. You may have a text file called
actors.csv, which looks as follows:
Surname, First Name, Street, Suburb, Postcode
Chopra, Priyanka, 200 Malabar Cres, West Bandra, 400050
Weaving, Hugo, 65 George St, Sydney, 2000
If you opened this file in a spreasheet program such as Excel, Numbers or Sheets, it might look like this:
Surname
First Name
Street
Suburb
Postcode
Chopra
Priyanka
200 Malabar Cres
West Bandra
400050
Weaving
Hugo
65 George St
Sydney
2000
Since CSV is such a simple format, it can be understood by virtually all data analysis programs from Excel to SPSS. If you are planning to export your data for statistical analysis, then CSV is likely to be the ideal format.
3.1.3 Describing the importing process
This import facility lets you import delimited text files:
CSV (comma separated values) file. Stores tabular data (numbers and text) in plain text. Each row of the file become a data record, while each comma-separated entry becomes a field.
TSV (tab separated values) file. Stores tabular data in columns and rows (as in a worksheet). Rows and columns are imported into records and fields.
The entries in the file are matched against entries in the database; unmatched rows can be added as new records.
The import process handles the following types of scenarios:
Pointer fields. Splits-out data into new record types linked with a pointer field (e.g. pulls out Authors or Place Names which are repeated for many records in the input data).
Multi-values. Manages multiple values in a column, multi-line text columns and handles imbalanced quotes and other typical CSV/TSV issues.
Misformatted data. Detects and reports line numbers for incorrectly formatted data to assist in correction. It can handle a wide variety of separators, long multi-line text fields with characters within fields, single quotes within double quotes and vice versa.
Geographic Data. Geographic data is accepted in WKT (Well Known Format); for example: POINT(x y). See here for more details.
Repeatable Fields. Multiple values for a repeatable field can be specified by separating the values with a | (pipe) symbol within the field. For example: 1,2,"3|4",5
Normalisation. In order to normalise the data (e.g. to extract a list of persons (entities) as records and then point to these person records rather than including names repetitively in the main data records), start by importing only those fields relating to the entities to be normalised. After import, the data will be redisplayed with the ID numbers for the extracted records, which can be used as a pointer field in the subsequent import of the remaining columns of data. You needn't assign all the columns as unassigned columns will be ignored. Duplicated records will be treated as you specified.
Disambiguation. When importing, Heurist tries to identify similar records which already exist in the database (a process known as disambiguation) and gives you the option of bookmarking one of these rather than making a new record.
3.1.4 Before You Begin
At a minimum, you must have a suitable record type structure defined in the database (it is possible to add addiitonal fields durign the import, but you at least need th record types and their connections) and a corresponding CSV/TSV file holding the entries you wish to transform into records.
Importing can be a complex business. It is important to clean up the data as much as possible in advance. The following provides some tips on how to prepare your data:
We recommend breaking very large files into manageable blocks of about two thousand lines.
Only one record type can be imported at each step of the process.
Have one row per entry, with each column containing a single element of data (split concatenated values into separate columns, and place notes about data items in a separate column, not appended to the data value).
The first line MUST contain column labels. Do it for your own sanity! The first line of your data also determines the expected field count.
Data rows must occupy a single line of data terminated with a linefeed: CRLF (Windows) or LF (Unix/Mac). Linefeeds within memo fields should be represented by CR only. Fields should be separated by tab or comma. Quotes may exist within unquoted fields, but within quoted fields they should be preceded by a backslash ( \" ). Fields containing the field separator should be enclosed in quotes. Editors such as Notepad++ (a free, open source Windows application) show tabs, CR and LF as symbols and can do global replacements on them.
Coded columns should use a consistent set of codes. In addition to your spreadsheet program, you may find OpenRefine a useful tool for checking and correcting coded columns, splitting fields, georeferencing, finding URL references and so on.
We strongly suggest editing the structure of the database to add any fields and terms that you will require for the import, before attempting to load the data. If you start trying to load data without the appropriate fields in place you will find it frustrating having to exit the process repeatedly to add fields.
If you have missing data for Required fields, you may find it convenient to set those fields to Optional before importing, then set them back to Required, then use Database > Structure > Verify to get a list of the records which need correcting. Alternatively, you can add some dummy value to the data, such as 'Missing', and search for this value after import.
The import process can be repeated on the file to extract multiple entities from different columns and replace them with record IDs which can be used in a subsequent insertion or update of records.
Please visit the page on Importing delimited text files on the Heurist network site for tips on successful import.
3.1.5 Delimited Text Importer Wizard
The Import Wizard takes you through a number of screens and steps to assist you in defining the import. (Read the screen instructions carefully. It might be a good idea to carry out a trial import with a small dataset to check that the result is as you expected.)
Set Data Source
These options are:
Select uploaded file. If you are importing a file you have imported before, select it from the dropdown. To clear this list, click [Clear All files].
Upload File. If you are importing a new file, select it using the [Upload File] button.
Paste Data. If you wish to use copied delimited text, paste it in the box below and click [Upload Data].
Set Import Parameters
For CSV files, before carrying out the import, you can set the import parameters (these settings are saved) as follows:
Encoding. Select the appropriate encoding.
Field Separator. Select the appropriate field separator: Comma or Tab.
Fields Enclosed In. Select the appropriate field enclosure: (', " or None).
Line Separator. Leave as Auto-Detect or select the appropriate line separator: Windows, Unix or Mac.
Multi-value separator. Select the appropriate multi-value separator: (e.g. | ; : /).
Date Format. Select the appropriate date format: European (dd/mm/yyyy or US (mm/dd/yyyy). Other date formats are possible and will be handled in the following wizard dialog.)
Click [Analyse Data] again to parse the expected results. This checks that the structure of your data matches what the Import Wizard expects. The header of the upload CSV (the first line of your data determines the expected field count) is checked against your import parameters, column names are extracted and encoding verified. The Import Wizard then attempts to convert the file based on your settings and displays the result (the expected input as rows (records) and columns (fields)).
Review the result and any error messages and update the source data if required. If you don't have Heurist Record ID (H-ID) value in your file, click on [Continue], else, specify the record that must be used to match the H-ID with already existing data.
:::info In this section you can also select any input column that contain dates (dd-mm-yyyy, mm-dd-yyyy or Iso standard) -- this allows the data to be parsed to extract consistency formatted date fields. :::
Once it's done click on [Continue].
Select Primary Record Type and Dependencies
The primary record type is the one represented by each row of the input file. Additional record types may be imported from selected columns prior to import of the primary, as determined by the dependencies shown. The creation of the primary record type from rows in the input file depends on the prior identification of other entities which will be connected via pointer fields or relationships. The tree below shows the dependencies of the primary record type determined from its pointer and relationship marker fields. Where an input entity matches an existing record, its ID value will be recorded in an ID field which can be used subsequently as a pointer field value; where no existing record is matched a new record is created and the new ID recorded. Check record types to be imported. Red indicates required pointer field.
3.1.6 The three importing steps
When the CSV/TSV data are loaded and that the record and connected entities are selected the import interface will take you through 3 important steps in order to correctly match and prepare your data for import:
Matching step which take care of verifying if data imported already exists inside the database and thus triggering the appropriate action (updating, deleting, etc.).
Fields to import step which define which columns of the imported CSV file will be imported into the database and in order to populate which field in the selected record type.
Insert/update step which take care of populating or updating the database given the chosen scenario.
Step 1. Matching
In the first step of the matching process you can choose what to match or to skip matching. Select a radio button:
Match on CSV Columns. To match import rows against existing records,select at least one Matching key column (later you can select for mapping) and ensure all selected key columns are allocated to a field. You can check (and scroll through) a sample of the field data to be mapped in the Values column. A new identification field will be created.
Matching sets this ID field for existing records and allows the creation of new records for unmatched rows.
Use Heurist ID column. (Only usable if H-ID column exists both in the CSV file and in the records to update inside heurist). In this case, the identification H-ID field (which is a field managed by the system) will be used.
Skip Matching (all new records). Skips the matching step (in this case only new records are created, one per input row).
Select the [Match on Columns] / Skip Matching button] (depending on the three previous cases). Matches are shown.
Step 2. Fields to Import
If all existing rows already match existing records (e.g. you may have already carried out the import successfully), then you can select the displayed Skip Update button to cancel the import.
The Import Summary box shows a mapping summary:
Existing. Number of matching records (that already exist based on selected matching columns). These will therefore be skipped.
New. Number of input rows for which no matching record has been found. These will therefore be added.
The following options are for matched or new rows:
Show. This displays the records on screen (click the Close (x) button).
Download. This downloads the records to a CSV text file.
The three matching, importing and inserting steps can work as an iterative operation if the spreadsheet data you are importing is a complex one. Therefore the import workflow allows you to progressively import columns which identify subsidiary entities (other Record Types linked through Record Pointers to the main record type you want to update or need to create data into) such as Place, Organisation, Collection, Series, Person, etc. The first step is to match identifying key fields and create new records from unmatched rows. The process starts with record pointers first and once all subsidiary entities have been matched and imported, you can import the primary entity type selected in the previous import phase.
Record IDs for the imported columns are added as column 1. Copy and save these data immediately if there are additional fields to import, to allow use of the record IDs as record pointers. Warning: you will lose the record IDs as soon as you start over, so save the data below to a file first.
If the displayed results are not what you expected, then go through the steps again (go back a step or click [Back to Start] if you wish to start again; all of your settings will be lost) and make any adjustments (including adjustments to the CSV or TSV file and/or Record Type).
Complete the Column to Field Mapping. Since new records are to be created, make sure you select all relevant columns; all Required fields must be mapped to a dedicated CSV column in order to proceed further. Click [Prepare] when ready (importing does not happen yet).
A message will appear if you haven't selected any fields other than the ones which are used to match records, so those are the only fields which will be set, and the result may be incomplete records. Click Proceed if you wish to continue, otherwise Cancel and review your settings.
Step 3. Insert/Update
In this step you carry out the update (this will update the database based on your settings so be sure this is what you wish to do).
Select an option on how you wish to treat data that already exists in a field:
Retain existing values and append distinct new data as multiple field values (existing values are not duplicated)
Add new data only if field is empty (new data ignored for non-empty fields)
Add and replace all existing value(s) for the record with new data
If you are happy to proceed with the import, click [Start Inset/Update]. You will be notified of the updates:
Click [OK] and close the window to exit the Import wizard. Review the imported records.
3.2 Zotero Bibliography
Zotero Bibliography Sync allows you to automatically synchronise a Zotero web library with the already existing bibliography structure within Heurist. It is especially powerful because it allows you to update bibliographic data from an active Zotero library, thereby saving time and effort in updating bibliography records within Heurist.
The synchronisation function looks for changes made since the last synchronisation, so it works fast even with a 20,000+ Zotero library once the initial synch has been done (which will take half an hour or so).
Heurist provides the following functions and capabilities for importing bibliographic data:
Automatic identification and disambiguation of imported bibliographic types.
Authors stored as person records allowing rich complementary data.
Series, Journals, Publishers stored as separate records to eliminate data redundancy.
Pre-defined domain profiles with collections of useful references, tags and searches.
To use the Bibliography Sync function, you first need to define a connection to a Zotero Library in
Design/properties/Synchronisation and Indexing. If this has not yet been done, in your database, you will be prompted to edit the settings that establishing such a Zotero connection. The relevant field is Zotero web library key(s) and IDs for synchronisation.
It should be noted that not all the zotero fields are synchronised with heurist bibliography record types. Moreover the synchronisation process will create automaticaly new records for Persons (author), organizations (Publisher), Places (publication location) and of course book references and so on. The Synchronisation is a one way process from a given Zotero collection to a Heurist database.
3.3 Heurist XML / JSON
3.3.1 Summary
Heurist XML / JSON allows data to be imported from an XML or JSON format that is specially tailored for compatibility with Heurist. When preparing data in this format, it is strongly recommended to first download the XML template. This is an XML document, following a Heurist-XML(HML) schema, that presents the core definitions of records that are necessary for proper functioning of your database. Following this template, you can design an XML document that can be easily read by Heurist. Once an HML or JSon-format is ready, select the file to upload from your desktop. Doing this takes you to a screen where the data is parsed and check. This screen enumerates the records to be imported and asks for final confirmation before the data is imported to create new records.
Click [Import Records] to start the import.
Contrary to the CSV/TSV import which allows a very refined way of updating or creating given field values with the use of matching and preparing steps, the XML/JSON import is a one time operation that imports a whole set of contents in one go. If the data is correctly formatted, as when exported from one Heurist database, it is a very fast and accurate way of importing data into another Heurist database (it can even download structure to accomodate the data provided the source database is a Registered database).
3.3.2 Import XML/JSON
Heurist will import HML exported from another Heurist database or from an external source which have been converted to HML format.
For Heurist database sources
Unless the source database structure is identical with the target, it should be registered first on the heurist master server which keep an index of unique identifiers for record type and fields in order to reuse it yourself or to be shared with other heurist users. You can register your database by going to [Design > Register].
Registration thus allows the target database to contact the source Heurist database in order to import (or update) the record (entity) type and field definitions it finds in the HML file, as well as permitting the inclusion of global conceptIDs in the HML.
For HML exported from a Heurist database, is normally set to indicate the source database. If it is set, synchronisation of definitions will be performed before the data are imported.
For non-Heurist database sources
To import a file generated from another source, eg. by transformation of an RDBMS to XML:
the XML file should conform to the template output from the target database using [Import > Download XML template];
the target database must contain definitions for all the record (entity) types and fields encountered in the XML file (in other words, only entity type and field codes defined in the XML template should appear);
the XML file should specify a Heurist database ID of 0 .
If a database ID is specified, synchronisation of definitions from that database will be performed before the data are imported. Since imported files will normally use a template for record types and fields exported from the target database, this is only useful for synchronising vocabularies and terms.
Record (entity) types and fields can then be specified using concept IDs (these will have a database ID of zero followed by the local ID (eg. 0-1234) for record types or fields defined locally in an unregistered target database.
Terms in the incoming data can be specified in one of the following ways which are evaluated in order:
first it looks for a valid local term ID.
If that is not found it tries to match it as a concept ID.
It then looks for an alphanumeric term applicable to the current field, and finally a standard code applicable to the current field.
If it gets to the end without finding a match, the value will be added to the database in the (first) vocabulary used by the field.
The XML Template
To create and import an XML file eg. to transfer data from another non-Heurist file or database, we strongly recommend using the XML tempalte which can be exported from Heurist using [Populate > Download template (XML)]. The template file contains full instructions for setting up the file. However, it is worth explaining the handling of record pointer fields in a little more detail.
To reference an existing record in the target database, the record number must be prefixed with H-ID- otherwise Heurist interprets the number as any identifier that matches the identifier filled in for another record in the import file, which may therefore be numeric or alpah/numeric.
This behaviour is quite intentional precisely to avoid making false connections (record IDs are database specific and cannot be known in advance unless re-exported and re imported, which is rendered unnecessary by our approach).
Note that inside the XML template, RECORD_REFERENCE may be replaced with a numeric or alphanumeric reference to another record, indicated by the tag. Note that this reference will be replaced with an automatically generated numeric Heurist record ID (H-ID), which will be different from the reference supplied. The reference supplied will be recorded in a field Original ID.
If you wish to specify existing Heurist records in the target database as the target (value) of a Record Pointer field, specify their Heurist record ID (H-ID) in the form H-ID-nnnn, where nnnn is the H-ID of the target record in the target database. Specifying non-existent record IDs will throw an error. The record type of target records are not checked on import; pointers to records of the* wrong type can be found later with [Admin > Verify integrity].
Example: I put in H-ID-2456 for a record pointer value:
if there is a record "2456", the record pointer is set to point to record "2456"
if there is no record "2456", it is reported as an error
if I put in any other type of value for the record pointer value eg. 2456 or wxyz or CallNo123456, it will look for a record defined in the XML with the value "2456" or "wxyz" or "CallNo123456" respectively, and set the record pointer value to the H-ID assigned to this record.
It will not try to second guess that "2456" is a valid record pointer value, because that is so database specific as to be almost certain to fail if there is no record with th especified ID, an error will be reported :::
3.4 KML
3.4.1 Introduction
KML is designed specifically for the import of bulk geospatial data into Heurist. In order to use this tool, first prepare a KML document in the standard format. Note that popular mapping tools such as Google Earth and Google Maps are able to natively export geospatial data in KML format.
3.4.2 Import KML
KML (Keyhole Markup Language) is a file format used to display geographic data in an Earth browser such as Google Earth, Google Maps, and Google Maps for mobile. KML uses a tag-based structure with nested elements and attributes and is based on the XML standard. All tags are case-sensitive and must be appear exactly as they are listed in the KML Reference. The Reference indicates which tags are optional. Within a given element, tags must appear in the order shown in the Reference.
Heurist will recognise the KML format and process the file, and prompt you for a record type. All records created by a single KML import will have the same record type.
Select [Choose File] and browse to select a KML file to import.
Click [Continue]. A summary of records to be imported is shown. When ready, click [Continue]. Heurist will recognise the KML format and process the file, and prompt you for a record type.
Select the record type and click Continue.
All records created by a single KML import have the same record type
4 Media Files - images, videos, audio and other files
4.1 Upload media files / images
Upload media files/images function, is designed for use by Database Managers only. It allows you to upload media files/images directly onto the Heurist server for use with a particular database. There are a range of allowable file formats/extension that can be uploaded in bulk in this way. As a Database Manager, you can select a media/upload folder in the relevant directory on the Heurist server. After selecting the target folder within this directory, Add Files from the desktop to upload. Once selected, click Start uploads to begin the process of copying these media files onto the Heurist server. Once completed, close the pane by clicking Finished.
4.2 Upload media from URL
Upload media from URLs function, is designed for use by Database Managers only, uploads a set of files specified by URLs, directly in the database. You can paste URLs and optional description in the area, CSV format is recommended. After pasting URLs or uploading CSV file, the URLs are checked and if the media files are supported, uploaded to the Heurist database. After uploading, assign each file to a record type and link it to the appropriate database entry by selecting file assignment.
4.3 Index external files
Index external files function, which is reserved for advanced users only, scans media folders and add missed to Media Files. Files have to be uploaded through Populate either using :
Function Upload media files/images
or by direct sftp access to the file_uploads directory (or sub-directories) on the server for larger files. Make sure the format of the extensions is supported by Heurist. Then, select the folders to scan. Click on [Proceed].
4.4 Create media records
Create media records function, is designed for Database Managers only, and is reserved for advanced users. It creates, updates and reads XML manifest files in the folders listed in Design > Properties and creates Digital Media records for all files uploaded to the database. Before, make sure to upload files through Populate (Upload media files/images). And make sure that the format of the extensions to scan is supported by Heurist. Click on "Continue" to synchronize the files.
4.5 IIIF Images
IIF (International Image Interoperability Format) provides a standard for image interchange widely used by museums, art galleries and others in the GLAM sector.
To enter an IIIF image, you need a File or Media URL field, when editing a specific record, enter the path of either:
a IIIF image with an url ending /info.json
a IIIF manifest with a name like manifest.json
Heurist will recognise these specific IIIF file and display them by using the embedded IIIF Mirador Viewer.
4.6 Process IIIF Manifests
Process IIIF Manifests function, is reserved for advanced users. It reads IIIF manifests and incluiding Annoftations, and creates or updates Annotation records in the Heurist database.
TODO: need more comprehensive documentaiton
5 Annexes
5.1 CSV Import Tips and Notes
5.1.1 Importing child records
Let's assume we have a Person Record Type with Child records linked fields such as Birth, Death, Life Event, Address association, etc. To import Address Association - which associates a Person with a Place for a particular date, date range or list of years - you must import Places to create Place H-IDs. But you must also import Persons to create Person H-IDs.
This may be tricky because the child pointer to these records may be a required field. But it needs to be done first in order to be able to create the child records.
5.1.2 Beware matching a repeating value...
Beware matching on a value which repeats as it can result in a new record for every value. For example, Address Association might be derived from a file listing an address for a particular person for each of 20 years in 20 rows. So one may have 10 rows with 17, First Street and 10 rows with 35, Second Avenue, and each of those rows has a different value in the year column.
What you want is TWO records, each with 10 years listed in a repeating YEAR field, not 20 records each with a year value and each address repeated in ten records. You should therefore ONLY match on Address (and Person). If you match on Year you will end up with 20 records, each with one year value, rather than 2 records, each with 10 year values.
5.1.3 Importing child records
Child records can be used to describe inherent and strongly dependent components of an entity, for example scenes in a frieze or painting, motifs in a scene, features of a building, worked edges on an artefact. They can equally be used to group rarely used attributes specific to a particular variant of an entity, for example pottery attributes for archaeological finds (where some finds are pottery, others bone, glass, stone or shell) - this is the case used here to illustrate the import of child records.
After defining all the fields for the Child Record type, you need a CSV file which either references the H-ID of the parent records, or a unique field or combination of fields in the parent records. In our case the Finds were imported from an Access database and the Find ID in the source database is included as Artefact ID. This allows it to be matched with Finds.Artefact ID (Access DB) in Heurist to obtain the parent Record Pointer.
The attributes to be imported into the child record will also be defined in the file. For categorised fields (a controlled list), we will use Heurist's Term List field type which may be represented in teh incoming data either as the labels or as the codes (foreign keys) used to reference the lookup tables in the source.
@TODO : check images on the previous paragrap in sharedocs document
To illustrate, let's define a test field Pottery type field with values "One", "Two" and "Three", which have numerical code 1, 2 and 3 respectively:
Here is the very simple test file imported by way of illustration.
Note that we use the code rather than the label (where exporting data from another software you may get either out of an SQL query depending on the way it is structured. In MSAccess, for example, some fields get joined with their lookup tables automatically and give you the label. Other softwares just give you the actual Foreign Key value in the field)
Artefact ID, Pottery type
244415, 3
This file is loaded using [Import > Delimited text (CSV/TSV)].
First, we select the Parent record type (which is called Finds in this test case) as the target entity type and carry out matching using a unique field or combination (Artefact ID in this case) in order to create the Heurist IDs for the parent records (Finds), either through finding an existing record and setting its ID or creating a new one and assigning a new ID. This step can be skipped if the file contains Heurist IDs for the parent records:
Once you've done that, change target to the child record type (Pottery information in this case) and match on a combination of fields which uniquely identifies each child record (these fields may include the parent record ID, that is Find H-ID in this case). Then select the field(s) you want to import (which ++must++ include the parent record ID, as this determines the parent appropriate to each child record):
In the data entry form for the record imported you will see the child record link (in this case we have not yet defined the full set of fields for the child record, nor the constructed title mask):
The child record identifies its parent and also shows the imported field(s). Notice that I imported "3" and it came out as the label "Three". That is NOT because Heurist connects numbers with their textual representation but because Heurist will look for the standard code if it does not find a matching label. If neither the label nor the code is recognised for one or more rows of incoming data, Heurist offers you the opportunity of adding unknown labels.
5.1.4 Importing relationships / markers
People often ask "How can I import a relationship marker". The short answer is "you can't" since relationship markers are just markers (and constraints) and contain no data. The long answer is, you don't import relationship markers, you import relationship records.
Importing relationship records from a CSV file
While relationships can be imported from an XML file, the easiest way is to create a CSV file containing the relationships, and import using the CSV importer. This minimally contains something to identify the source record (eg. names or the Heurist ID) and the target record, adn the type of relationship. Dates and other attributes eg. notes or bibliographic references or degree of certainty, can also be provided.
Source Name, Source First name, Relation type, Target Last name, Target
First name , Start date, End date \
Dupond, Michel, is husband of, Dupont, Anne, 1512, 1531\
Dupont, Bernadette, is wife of, Dupond, Jean,,\
etc.
The direction in which the relationship is defined does not matter provided the right term is used. Relationship type can either be directional, as in the case of isChildOf and isParentOf, or non-directional eg. isRelatedTo
Use [Import > Delimited (CSV/TSV)]:
Identify the target record type as Relationship record.
Relationship records might be marked as a hidden record type, in which case they will not show up in the options. Go to Design > Record types and set them as visible.
Match on the two source columns (Source Name and Source First Name in this case, or other columns that will identify the source record, for example the ID or title etc.), then a match on the two target columns. This will create appropriate Heurist ID columns (if the Heurist IDs are already in the file these can be selected).
Set the generated Heurist IDs to match the Source and Target record pointers, and the relationship type to match the relationship type field.
Finally, import the data into the Relationship records.
The imported relationship records will appear in any relationship markers whose constraints they fit.
Note: Relationship Markers do not contain any data. Nothing! These are markers that have two functions:
Show Relationship records that match the marker (type of source, type of target, type of relationship)
Show where you want to create relationships and constrain possible relationships (target type (s), relationship types)
If you put a Relationship Marker in the source type records, and another in the target type records, and if the relationships are either non-directional eg. isRelatedTo (applies in both directions) or the inverse of one-another (eg isChildOf and isParentOf), the relation will show in both records with the appropriate terms, for example:
5.2 Detailed mapping and use of KML and spatial data
5.2.1 KML Field Definitions
This table shows how the data is mapped into Heurist; it lists the KML tags that Heurist recognises as record details, and the bibliographic data fields that they are imported to.
Contact the Heurist Network Association for the full list of KML Field Definitions for the XML file to determine how the data is mapped into Heurist.
Heurist attempts to import each as a separate record.
KML tag
Heurist detail field
Title (detail type #160)
Location (#181)
Contact information (#309)
Start Date (#177)
End Date (#178)
Date (#166)
Geographic object (#230)
Shared scratchpad
It is possible to specify Heurist-formatted data in HXTBL format between KML's tags. For example:
...
...
Archaeological Computing Laboratory
Laboratory
...
...
Heurist will add fields of type #160 (Title) and type #203 (Organisation Type) to the record corresponding to this .
Ch 06a: Importing and matching references (worked example)
Ian Johnson, updated 29 May 2026
Background
This section gives a worked example of importing two sets of references (Primary and Secondary) for inscriptions from a spreadsheet derived from Zotero data. The example comes from the IDENK project (Idenk.net) based at the EFEO, courtesy the poject director Arlo Griffiths [REQUEST AGREEMENT, I am sure it will not be a problem]
Basic structure
Inscription records contain two record pointer fields (Primary references and Secondary references)
These point to Bibliographic reference records
Bibliographic reference records contain
a record pointer field to a bibliographic record (Book, Chapter, Journal Article, Thesis etc.) derived from the Zotero library
pagination information (a text field containing page numbers, illustration or other information about sections of the document)
Bibliography records are of various types (Book, Chapter, Journal Article, Thesis etc.)
The bibliography records are identified by strings such as Adams1912_01 (not shown in the view above). These are identifiers which have been filled in the Zotero Short Title field in a large Zotero library (21,000 references). These are referred to as ZSTs.
*One could also use the Zotero key field, which is automatically populated with an 8 alphamnumeric hash key which is statitically unique and cannot be edited - it is this key that we use to link our internal bibliographic records back to their Zotero origin records.*
The steps are as follows:
The first batch were already in the Heurist database so these identifiers are exported to a spreadsheet. Later batches will be imported from a spreadsheet used to collect data 'in the field'
If not already separate in the spreadsheet, use Libre Office Data > To columns to split the ZST and the pagination reference (page numbers and or illustrations)
Deduplicate the rows in the spreadsheet (Dat > Duplicates) based on the ZST and pagination. Each row will then be a reference to a particular place in a particular bibliographic record.
Import into Bibliographic reference records
Split the original spreadsheet (before deduplication) into two CSV files, one for Primary and one for Secondary references
Import each of these files into the Inscriptions table, the first into the primary reference field, the second into the secondary reference field.
These fields are record pointer fields pointing to Bibliographic reference records, so the textual ZST + pagination value is first matched with the values in the database and this generates the ID of the Bibliographic reference records created in the previous steps, and ut us this ID which is inserted into the record pointer fields.
After updating the bibliographic records (Book, Chapter, Journal Artivle, |Thesis etc.) from the Zotero library (Populate > Zotero Bibliography sync we need to relate the Bibliographic reference records to the bibliographic records by matching the ZST in the first with the Zotero Short Title field in the second, using Recode > Foreign Key match.
From scratch
Delete all Bibliographic references. This also deletes the pointers to them from Inscriptions.
Preparing the spreadsheet
Batches 2 and 3 will already have their much more comprehensive spreadsheet, see later.
Export all the existing ZST references for Primary and Secondary refs (not Surrogates)
to CSV using CSV Primary Secondary refs custom report:
Open in Libre Office (the delimiter is tab, not $ as shown)
Highlight ZST column, Data > Text to columns using the colon ( : ) as a delimiter:
You now have the original ZST+pages value and separate ZST and Pagination values in the last two columns (some have no pagination so the last column will be empty).
Now deduplicate on the ZST Pages column in LibreOffice (Data > Duplicates).
Rather than child records we will point multiple inscriptions to common Biblio Reference records which include a page range. Note that editing these records can corrupt other Inscription entries which point to it if the change is such as to change the reference, since the Bibliographic reference records are to a specific place in a particular bibliographic entity.
The alternative is the use of child records and significant duplication (1 in 4 approx). Using independent bibliography references is altogether simpler to deal with apart from the slight drawback above.
Preparing the batch X spreadsheet
To document when I have the final spreadsheet
Loading the bibliographic references and linking
Load into Heurist with Populate > CSV.
Select INScriptions for H-ID as these records relate to data in the Inscriptions
However, first choose Bibliographic record pointer as we want to create bibliography records and then reference them in Inscriptions.
Skip matching as we will import all the records in this first batch since there are currently no Biblio reference records and we have deduplicated.
For subsequent additions you will need to match with existing values
Note: after deduplication we have 1114 Biblio references, these examples were pre deduplication
We now have 1114 bibliographic records like this:
Now split the incoming original spreadsheet into Primary (n=398) and Secondary (n=1076) references based on PRI and SEC in the first column.
Save as two CSV files. Load each in turn.
Select Inscription as the primary type and Primary refs or secondary refs as the dependency (these images are for the Secondary refs)
It will first ask you to match the Bibliographic references in order to set the H-IDs for those references which are to be inserted into the Inscription records.
That sets the IDs of the Bibliographic reference records.
Now select the Inscription records which are to be updated.
Click on Use H-ID (this was in the original file and referenced the INScriptions)
Existing: 191 New: 0 tells us that there are 191 Inscriptions (of the 398) which have Primary bibliographical data (for the Secondary references it's Existing: 282 New: 0)
We import the Primary references H-ID into the Primary refs > record pointer field (later, the Secondary references H-ID into the Secondary refs > record pointer field):
Primary references:
Secondary references:
Prepare, then Start Update:
Primary references: Secondary references:
and all looks good:
Connecting Bibliographic reference records with bibliographic entities
Now we have to connect our Bibliographic reference records with the appropriate bibliographic records imported from Zotero. We must do this for each of the reference types used since we cannot match across multiple tables.
For books:
and for each of the other types:
These are Bibliographic reference record which do not match up with a Zotero record using the ZST, and in most (all?) cases these ZST do not exist in the database except in these records. This needs to be checked individually.
Ch 06b: IIIF Manifests, Canvases and Annotations
This guide describes the IIIF features provided by Heurist for creating, importing, viewing, editing and exporting IIIF Manifests, Canvases and Web Annotations.
Heurist supports two main workflows:
Use Heurist as an annotation layer over existing IIIF Manifests (annotation overlay mode). The external provider keeps ownership of the source Manifest and Canvas identifiers. Heurist stores and publishes local annotations.
Use Heurist to manage the Manifest (full management mode). Heurist stores Manifest, Canvas and Annotation records and generates a IIIF Presentation API v3 Manifest from those records.
Heurist also provides a dynamic IIIF server for ordinary record sets and registered media files, and can render external IIIF files and Manifests. In this sense it can act both as a IIIF client and as a IIIF server.
1. Preparation
1.1 Import the required definitions
Before using the IIIF annotation and Manifest tools in an existing database, import the new definitions from the
Heurist_Core_Definitions database using Design > Browse templates. Heurist will prompt you to do this if you attempt to process Manifests without the required definitions.
The new record types are in the Documents group. It is enough to select IIIF Annotation. The related record types IIIF Manifest and IIIF Canvas are downloaded alongside it.
The important record types are:
IIIF Annotation (
RT_IIIF_ANNOTATION, concept code
2-109)
IIIF Manifest (
RT_IIIF_MANIFEST, concept code
2-110)
IIIF Canvas (
RT_IIIF_CANVAS, concept code
2-111)
These definitions include fields for IIIF identity, original/source IIIF identity, Manifest links, Canvas links, annotation state, selector type/value, annotation JSON and related metadata.
1.2 Remove obsolete duplicate fields in old databases
Some older databases may contain a duplicated field named IIIF Anotation 2 with:
local ID:
1106
concept code:
2-1098
This field is not used by any current IIIF record type. Remove it before using the new IIIF workflow, especially if it causes confusion in forms or import checks.
1.3 Recommended checks before testing
After importing definitions, check that the database contains the three IIIF record types above and that Browse templates no longer shows missing IIIF definitions in the Core definitions database.
For testing, start with a small Manifest first. A large external Manifest may fail for reasons unrelated to Heurist logic, such as network timeouts, remote annotation-list delays, or unavailable image services.
2. Key concepts
2.1 Manifest
A Manifest is the IIIF object that describes a digital object, such as a manuscript, book, image set or media collection. In Heurist, a Manifest may be:
a registered external Manifest file or URL;
a managed IIIF Manifest record;
a dynamic Manifest generated from a record set or a single registered media file.
Managed Heurist Manifest output is generated as IIIF Presentation API v3. A registered IIIF Manifest file becomes managed only when an IIIF Manifest record references that file. If no such record exists, Heurist treats the registered Manifest file as an external/source Manifest and can use it as an annotation overlay target.
2.2 Canvas
A Canvas represents one viewable unit in a Manifest, for example a page, image, video or audio item. In full management mode, Heurist stores each Canvas as an IIIF Canvas record. Each managed Canvas normally points to a registered file or registered external media URL.
In annotation overlay mode, Canvas records are not imported or managed by Heurist. Instead, annotations remain linked to the original Canvas URI from the source Manifest.
2.3 Annotation
Annotations are stored as IIIF Annotation records. They may be created or edited in Mirador, mainly for defining the annotation area and initial text, or in the Heurist record editor for annotation attributes, which can be extended to support searching and custom reporting within Heurist.
Annotations store:
text body / summary;
motivation, such as commenting;
language;
original Canvas target URL;
managed Canvas reference when applicable;
selector type and selector value;
raw IIIF/Web Annotation JSON;
state, such as imported, Mirador-created, Heurist-created, modified, obsolete or removed.
3. Manual creation of a managed Manifest
Manual creation is used when you want Heurist to own and generate the Manifest rather than only overlay annotations on an external Manifest.
3.1 Create the Manifest record
Create a new IIIF Manifest record. Fill in Manifest-level metadata such as title, description and copyright/rights. These fields are used when Heurist generates the v3 Manifest output.
A managed Manifest can be empty. An empty managed Manifest still returns valid IIIF Presentation API v3 JSON with
items: [], so viewers should not normally show a technical error.
3.2 Add Canvases one by one
Create IIIF Canvas records and link them to the Manifest. Each Canvas may reference:
a locally uploaded registered file;
a registered external media URL;
an image served by a IIIF Image API;
other supported media such as audio or video where configured.
The order of Canvas references on the Manifest record defines the order in the generated Manifest. The order can be changed within Heurist data entry by dragging the Canvas references up and down.
3.3 Add or edit annotations in Mirador
Open the managed Manifest in the Mirador Viewer. Use Mirador's annotation tools to add annotations to the selected Canvas. Heurist stores the annotation as an IIIF Annotation record and links it back to the relevant Canvas and Manifest context.
The internal Mirador viewer uses the default annotation lookup scope
canvas, which reads annotations from
/api/{db}/annotations. A Manifest-scoped endpoint is also available as
/api/{db}/annotations/{manifestRecID} when
annotation_scope=manifest is requested.
3.4 Edit annotations in the Heurist record editor
Annotations can also be edited directly as Heurist records. This is useful for correcting text, language, motivation or metadata.
Be careful when editing selector information manually:
Selector type and selector value must remain consistent.
A rectangular fragment selector and an SVG selector are not interchangeable.
If the selected area is edited incorrectly, Mirador may display the annotation in the wrong place or fail to display the region.
In general, use Mirador for changing the selected area and use Heurist record editing for textual and descriptive metadata.
3.5 Open the Manifest, Canvases and Annotations from the Record View panel
From the IIIF Manifest record view, open the Manifest either as raw/generated IIIF content or in the Mirador Viewer.
Related records can also provide navigation back to the Manifest context:
IIIF Canvas records may include a link to open the referenced Manifest in which the Canvas is used.
IIIF Annotation records may include a link to open the referenced Manifest, so the annotation can be viewed in its wider Manifest context rather than as an isolated record.
IIIF Canvas records can also be opened independently, in the same way as any Heurist record with a file field. This is useful when checking a single page/image/media item before opening the full Manifest.
For internal Mirador viewing, Heurist passes
omit_annotation_pages=1 to the generated Manifest URL where needed. This prevents the same database annotations from being loaded twice: once from embedded Manifest annotation-page links and once from Mirador's annotation endpoint.
3.6 Add Canvases in a batch — planned feature
A planned batch action will allow users to select one or several ordinary records that already have file fields and create Canvas records from those files. This is intended to make managed Manifest creation faster for large image sets.
Until this is implemented, add Canvas records manually or import/process an existing Manifest in full management mode.
4. Import or process an existing IIIF Manifest
Use Process IIIF Manifest to work with a registered or uploaded IIIF Presentation Manifest. A Manifest can be registered as:
an external IIIF Presentation Manifest referenced by a File field;
a JSON Manifest uploaded to Heurist as a File field.
The default mode is Full manifest management, which creates or updates an IIIF Manifest record, imports IIIF Canvas records and imports available IIIF Annotation records.
Annotation overlay is different: it imports annotations only. It does not create an IIIF Manifest record. The registered Manifest file remains the source Manifest and Heurist stores local annotations against the original Canvas URIs.
4.1 Annotation overlay mode
Use Annotation overlay when the external Manifest remains the authoritative source for Canvas structure.
In this mode:
only IIIF Presentation API v3 Manifests are supported;
the source Manifest and its Canvas list remain owned by the external provider;
Heurist does not create an IIIF Manifest record;
Canvas identifiers are preserved from the source Manifest;
annotations are imported into Heurist and linked to the original Canvas URIs;
when
/api/{db}/iiif/manifest/{obfuscatedFileID} is requested, Heurist can output a v3 overlay Manifest by replacing source
Canvas.annotations with Heurist AnnotationPage links;
local Heurist annotations are preserved on re-import/re-processing when they have been edited locally.
Do not use this mode for IIIF Presentation API v2 Manifests. For v2 source Manifests, use full management mode. If a managed IIIF Manifest record already references the selected registered Manifest file, annotation overlay mode is not available because the file is already under Heurist management.
4.2 Full manifest management mode
Use Full manifest management when Heurist should manage the Manifest structure.
In this mode:
Heurist creates or updates Manifest, Canvas and Annotation records;
the existence of the IIIF Manifest record is what marks the registered Manifest file as managed;
Heurist owns the generated Manifest output, Canvas order and Canvas metadata;
media may still be external registered resources or local uploads;
media should be stored in Heurist where referenced resources are not held by a stable long-term repository or institutional service;
Manifest-level metadata can be edited in Heurist;
Canvas order comes from the Canvas references stored on the Manifest record;
annotations are linked to managed Canvas records;
generated IIIF output uses Heurist Canvas API URLs.
This is the preferred mode for IIIF v2 source Manifests, because the overlay workflow is v3-only.
4.3 Re-import / re-processing behaviour
On re-import, Heurist attempts to update imported records while preserving local work. Records that have been changed in Heurist or Mirador are preserved by default and reported separately as preserved local records.
The report includes:
managed Manifest record ID, or
not created for annotation overlay;
total Canvases found;
Canvas records added, updated, unchanged or preserved;
total annotations found;
annotation records added, updated, unchanged or preserved;
issues encountered during import/processing.
4.4 Thumbnails
The import tool can create thumbnails for annotation records. This is useful for browsing annotations in Heurist, but it is slower because it may need to access remote images or render selected regions.
5. Add annotations for an arbitrary registered file or URL
You do not need a managed Manifest before annotating media.
You can open the Mirador Viewer for any registered media file or supported registered URL. Heurist dynamically creates a single-canvas Manifest for the media and lets you add annotations. These annotations are stored in Heurist against the Canvas URL used for that file.
If you later add the same file to a managed Manifest, the annotation can be preserved because the Canvas identity is based on the registered file's obfuscated ID. This allows annotation work to start before the final Manifest structure is prepared.
Typical uses:
annotate a single image before adding it to a larger Manifest;
annotate a registered external IIIF image;
test annotation behaviour on one file before importing, processing or building a large Manifest.
6. Viewing in Mirador
Heurist provides a Mirador Viewer for:
a managed IIIF Manifest record;
a registered external Manifest file;
a single registered media file;
a dynamic Manifest generated from a query or selected record set.
Registered Manifest files are opened through
/api/{db}/iiif/manifest/{obfuscatedFileID}. If an IIIF Manifest record references the file, the API returns the managed Manifest generated from Heurist records. Otherwise it returns the source Manifest: v2 sources are returned as-is, while v3 sources can be returned with Heurist annotation-page links overlaid.
The viewer supports two annotation lookup scopes:
annotation_scope=canvas — default. Shows all annotations that target the same Canvas URL.
annotation_scope=manifest — shows only annotations linked to the current Manifest record.
For internal Mirador viewing, Heurist avoids duplicate annotations by passing
omit_annotation_pages=1 to generated Manifest URLs where needed. External IIIF consumers can receive normal
Canvas.annotations links when this parameter is not used.
7. Dynamic Manifests via Export IIIF
Heurist can generate IIIF output dynamically from ordinary record searches and file selections. This is useful when you want to view or share a record set without creating a permanent managed Manifest record.
7.1 Single registered media file
A single media file can be opened in Mirador or exported as a IIIF Manifest by using its registered file obfuscated ID. Heurist wraps the media in a single-canvas IIIF Presentation API v3 Manifest.
Useful for:
quick viewing of one image, audio or video item;
adding annotations to one registered file;
testing IIIF output for one file.
7.2 One ordinary record with media files
When a record contains one or more suitable file fields, Export IIIF can generate a Manifest whose Canvases correspond to the media files linked to that record.
Useful for:
records that represent objects with several images;
quick Mirador viewing without creating explicit Canvas records;
public sharing of record media as IIIF.
7.3 Several ordinary records with media files
When the current record set contains multiple records with suitable media, Export IIIF can generate a Manifest with one or more Canvases from those records, subject to the export limit.
Useful for:
search results containing image records;
temporary collections;
comparing several media records in Mirador.
7.4 One registered IIIF Manifest in the record set
If a record set contains one registered IIIF Manifest and no generated media Canvases, Heurist can return that Manifest directly through the IIIF API.
Useful for:
opening a registered external Manifest through Heurist;
keeping a registered Manifest discoverable as a file in a record;
testing external Manifest access.
7.5 Several registered IIIF Manifests in the record set
If a record set contains several registered IIIF Manifests, Heurist can generate a IIIF Collection that references those Manifests.
Useful for:
publishing a set of related Manifests;
opening several Manifests together in Mirador;
grouping imported, processed or external Manifests without merging their Canvas structures.
7.6 Mixed record set: registered Manifests and media files
If a v3 dynamic export contains both registered Manifests and ordinary media Canvases, Heurist can generate a Collection. Registered Manifests become Manifest items in the Collection; generated media Canvases are grouped into a generated Manifest item.
Useful for mixed search results where some records already contain IIIF Manifests and others contain image/audio/video files.
7.7 IIIF v2 output policy
Heurist no longer generates IIIF Presentation API v2 output. Dynamic export and managed Manifest output are v3-only. Heurist can still import v2 and hybrid v2 source Manifests in Full manifest management mode and then publish them as generated v3 Manifests.
8. Recommended workflow examples
8.1 Annotate an external v3 Manifest without taking over its structure
Register or upload the v3 Manifest JSON.
Open Process IIIF Manifest.
Select Annotation overlay.
Import/process annotations.
Open the registered Manifest file in Mirador. The viewer uses
/api/{db}/iiif/manifest/{obfuscatedFileID} and the annotation endpoint.
Add or edit annotations.
Use the same API URL when external viewers need the v3 source Manifest with Heurist AnnotationPage links.
8.2 Import a v2 Manifest with many Canvases and annotations
Register or upload the v2 Manifest JSON.
Open Process IIIF Manifest.
Select Full manifest management.
Import/process Canvases and annotations.
Inspect the report for failed remote annotation lists or unavailable image resources.
Open the managed Manifest in Mirador.
If the v2 Manifest is very large, test first with a trimmed Manifest containing a few Canvases.
8.3 Start with one image and later build a Manifest
Register or upload an image.
Open the image in Mirador.
Add annotations.
Later create a managed Manifest and add that file as a Canvas.
The annotation can be preserved because it targets the file-based Canvas identity.
9. Troubleshooting
The import widget says required definitions are missing
Import IIIF Annotation from
Heurist_Core_Definitions. The related Manifest and Canvas record types should be imported with it.
The database contains an old field named “IIIF Anotation 2”
Remove the obsolete duplicate field with local ID
1106 and concept code
2-1098. It is not used by the current IIIF record types.
Overlay mode rejects a v2 Manifest
This is expected. Annotation overlay mode is v3-only because it stores annotations against original v3 Canvas URIs and can publish v3
Canvas.annotations AnnotationPage links. Import v2 Manifests in Full manifest management mode.
Overlay mode is disabled for a selected registered Manifest file
This means an IIIF Manifest record already references the selected registered Manifest file. That file is already managed by Heurist, so use Full manifest management mode.
Mirador shows duplicate annotations
Use the internal Heurist Mirador viewer, which passes
omit_annotation_pages=1 for generated Manifest URLs where required. This avoids loading the same annotations both from Manifest
Canvas.annotations and from Mirador's annotation endpoint.
Import fails on a very large Manifest
Try a small trimmed Manifest first. Failures may be caused by remote annotation-list access, timeouts, malformed source JSON, unavailable image services, or network interruptions.
10. Summary of ownership by mode
Feature
Annotation overlay
Full manifest management
Supported source Manifest version
v3 only
v2 and v3
Source Manifest ownership
External provider / registered file
Imported into Heurist management
Generated Manifest output
Source v3 Manifest with Heurist AnnotationPage links when requested through the IIIF API
Heurist managed v3 output
Canvas list ownership
External provider
Heurist
Canvas identifiers
Original source Canvas URIs
Heurist Canvas API URLs
Canvas records created
No
Yes
Annotation records created
Yes
Yes
Manifest metadata editable in Heurist
No managed Manifest record is created
Yes, used in generated output
Best use
Add Heurist annotations to an existing v3 Manifest without creating a Manifest record
Build or take over a Manifest in Heurist
Ch 06c: Omeka-S to Heurist
Omeka S is a configurable database (there is an older version Omeka Classic). It is much more complex to set up and much more limited, although it does have some functions in the semantic web area which we don't yet address and extensive tech documentation, having been defined from scratch after a decade of Omeka Classic, and is therefore easier for programmers to extend with add-on modules. There is also an Omeka (either version) to Datacrate conversion and Heurist to Datacrate conversion developed in Python by Peter Sefton at UTS - you can find Datacrate on github - which might form the basis for an alternative pathway.
Please note that the migration from Omeka S to Heurist was developed before 2020 and may not operate 'out of the box'/
Converting from Omeka S to Heurist
The following table shows the correspondences between structures defined in Omeka S and structures defined in Heurist:
Omeka S
Heurist
Resource_class
defRecTypes
Resource_template_property
defRecTypeStructure (order, altlabel, requirements and data_type?)
Property
defDetailTypes
Resource
Records
Value
recDetails
Conversion
Since data_type is not defined in Resource_template_property (it was empty in def19 databases), it is necessary to detect type for every property.
++Resources++: where value.value_resource_id IS NOT NULL
++Terms++: look at tables with the same name as property and number of distinct values <100
++Blocktext++: where number of long values is considerable length(value.value)>100
Get all properties in use
SELECT p.id, p.local_name, count(\*) FROM value v, property p
where v.property_id=p.id group by p.id, p.local_name order by p.id
Get properties in use by record class
SELECT distinct r.resource_class_id, p.id, p.local_name FROM value v,
property p, resource r where v.resource_id = r.id and
v.property_id=p.id
Order by r.resource_class_id, p.id
As a result, you need to create following CSV tables.
For terms
Property id: list of enum properties uses the same vocabulary
Table name: takes terms from this table, don't worry if value is missed in this table it will be added to target vocabulary
Vocab name: name of vocabulary to be added to heurist
Resource class ID: check properties for these class only. (in OMEKA some fields are inconsistent for its types for different classes)
Property ID
Table Name
Vocab Name
Resource Class ID
202
fonctions
fonctions
155
"223,245,325"
pays
pays
283
causes-fin-brevets
brevet cause fin
291
genres
genres
329
types-adresses
types de adresses
346
typesdeproces
types de proces
"290,383"
roles
roles
For all fields:
$config = <<<'EOD'
rty
id
local_name
dty_Type
dty_ID
ptr/vocab Explanation
7
date
date
9
252
birthdate
date
35
isReferencedBy
blocktext
131
nick
freetext
95,110,111
143
surname
freetext
1
map property 143 to heurist 1 for classes 95..
150
143
surname
resource
16
map property 143 to heurist 16 for class 150
230
parrain
resource
95
125
gender
enum
20
202
agent
enum
6255
Classes by records
SELECT resource.resource_class_id, rc.local_name,count(\*) FROM
resource, resource_class rc
where resource_class_id=rc.id group by
resource.resource_class_id,rc.local_name
Conversion notes (for developers)
I will do mapping their ResourceClass/Property to Heurist Rectypes/Fields
Enumeration types are vague in their system. If some of properties have table of the same name (for example property genre has table genres this property considered enumerated)
Import Resource/Values to Records/recDetails
DEFINITIONS: Map existing Heurist record types/fields to Omeka resource classes/properties.Omeka database does not keep any information about its database definitions just two tables that refers to resource/properties of RDF models (url of xml that describes these models are in Vocabulary table).
Example:
Resource class Agent (id 95, vocab_id=4) refers to Agent in http://xmlns.com/foaf/0.1/
Property Genre (vocab #6) refers to http://dbpedia.org/ontology/genre
Manual matching Omeka->Heurist: Resource class->Rectypes Property->Field type
Store RDF name (like foaf:Person OR dbo:Genre) in some field of defRectype, defDetailTypes tables OR keep matching in external file Omeka ID->Heurist ID, or RDF name->Heurist concept code I believe it is much cleaner to store such data in the database, this then allows us to use it directly in a future RDF export. Every time we use files we end up with problems eg. of synchronisation, referential integrity etc.
DATA: Import Omeka resource/value tables into Heurist Records/recDetails
Ch 07: Using the database (find, filter & view
[Explore] is the workhorse function that allows you to make use of the data recorded in a database. The core function of Explore is filtering the database to isolate a subset of the database to which some sort of listing, analysis, visualisation or export will be applied (filter also acts as a simple search to locate information to look through eg. a reference, web bookmark or images). This workflow, from filter through results list or subset to reading, visualization, analysis and output, is represented in the left-to-right flow across the Explore screen :
filter building and saved filters on the left
results listing in the middle
various visualisations and outputs on the right. This is the starting point for all information retrieval.
1. Overview of the [Explore] menu
1.1. Filters
Here you can find pre-programmed filters for viewing particular records in the database.
1.1.1 Recent | All by date:
[Recent]: View the most recently added or modified records, with the most recent at the top. This is useful for fetching the records you are currently working on.
[All by date]: View all the records in the database. This is useful for browsing small databases.
1.1.2.Entities
[Entities]: Filter the database by record type.
Displays records sorted by entity type (favourites or ordered by most used). For example, you might wish to see all the Persons in the database, all the Places, all the Books or all the Events.
1.1.3. Saved filters
[Saved Filters] give an access to filters or faceted searches you have created yourself and previously recorded for re-use (frequently used or used in website publication).
Heurist allows the saving of filter criteria which become entries in a tree of saved filters, accessible through [Saved filters and Rules] menu entries, and in a dropdown below the [Filter] button.
Saved filters can be simply a predefined filter which generates a given subset of the database for a specific purpose (eg. sets of things you need regularly, perhaps sorted in a specific order, or a list to be displayed in a website), or they can be facet filters which provide a guided pathway allowing interactive exploration of the database through the display of subsets with frequency of occurrence according to the selections made.
Saved filters (simple or facet) and rules can also be created directly from the list of saved filters by clicking on the rollover icon or right-clicking on the list. The dropdown menu also allows the creation of folders within the list, editing and deletion, and other functions. Filters can be moved by drag and drop.
Note also that saved filters and rules are organized by workgroup, to allow database managers to create different sets of filters for different groups of users – for example the filters needed by volunteer data collectors or filters to be displayed on a CMS website (a Website Filters workgroup is defined by default for this purpose).
1.2. Build
1.2.1. Filter builder
[Filter builder] Open a wizard which can be used to create a custom filter, which selects records from the database that meet certain criteria.
For example, you may wish to see all living Persons in the database, or all the Places that lie within a particular region.
The Filter builder provides an easy way of building queries of moderate complexity, hiding the complexity of writing filter strings. Simple searches, such as a partial string match on title, can be entered directly in the filter fields or constructed with the Filter Builder.
1.2.2.Facets builder
[Facets builder] Open a wizard to build sophisticated multi-level facet filters and rulesets.
This wizard configure a faceted search, in other words an interactive filter (which will be familiar from online shopping sites). For example, you may wish to search for People by surname, while also having a time-slider to filter by birthday at the same time. Using the facets builder, you can decide which aspects of a record you would like to use for filtering (e.g. surname), and decide what kind filtering interface you would like to use (e.g. a searchbox or dropdown).
The Heurist system for building facet filters is not restricted to building facets on the attributes (fields) of a single selected entity type. It can drill down into the connections between entity types to allow selection on the attributes of related records at several levels removed. The choices are made from a treeview of attributes which can be expanded to view the attributes of connected entity types.
The facet builder can also apply rules to traverse the network of connections to find entities which are connected to the results of a facet filter. Rulesets can be created and used independently.
These queries allow a range of sophisticated instant analyses, without programming, along the lines of “select all the organisations which have published books written by female authors who have degrees from a University located in London”.
Facet filters can be embedded into websites generated by the Heurist CMS.
1.2.3. Save filter for re-use
This tool saves the current filter (simple or faceted) into the tree of saved filters for reuse.
Heurist allows the saving of filter criteria which become entries in a tree of saved filters, accessible through [Saved Filters] and Rules menu entries (cf., and in a dropdown below the Filter button.)
Use of workgroups Saved filters and rules are organized by workgroup, to allow database managers to create different sets of filters for different groups of users – for example the filters needed by volunteer data collectors or filters to be displayed on a CMS website (a Website Filters workgroup is defined by default for this purpose). :::
1.3. Advanced
1.3.1. Rules
Rules are expanding search results to connected entities. In other words they allow you to select interrelated sets of records of different types from the database.
For example, when searching for people in the database, you may wish to display the record for a Person's spouse or place of residence as well as the record for the Person themself. To do this, you would create a ruleset which defines exactly which related records to retrieve when you search for people. These rulesets can be used in conjunction with custom filters or faceted searches.
1.3.2. Set as subset
:Set as subset saves the current set of records as a subset, which can then be filtered or manipulated further.
This menu item restricts further filtering to the current result set. This can be useful to isolate a specific set of records for further filtering, visualisation or analysis eg. all the records from a specific collection or all the works by a specific set of authors. Once set, the subset can be cancelled with the undo icon which appears at the end of the menu item.
2. Build and save a simple search or filter
2.1. The search box
At the top of the Filtered Results pane in the Explore Menu, there is a searchbox : you can use it to do simple searches of the database, but it also drives Heurist's advanced filtering features.
If you are an advanced user, you can learn to use Heurist JSON Query Language, and design powerful, customisable queries quickly and precisely (although it is much easier with teh Fitler Builder).
Directly search for records using a range of modifiers: tag: , type: , url: , notes: , owner: , user: , field: and all:.
For example, to search for tagged records in the database, enter either tag : string or tag = string in the Filter box. For example, tag : Database (any tag including the string ‘Database’) or tag = Database (matches Database but not ‘Databases’ or ‘Database Management’). Tags are not case sensitive (i.e. 'database' = 'Database').
If search terms include a space, enclose them in single or double quotes (e.g. tag : ’Database Management’ ).
To find exact matches, use the = operator (e.g. title = xxx ). You can also use the greater than (>) and lesser than (<) operators if you are filtering by a numerical or date field (e.g. year < 2007 would find records from before 2007).
To find records that include either of two search terms, use an uppercase OR (e.g. timemap OR “time map”).
To find records with geographic objects that contain a given point, use latitude and longitude (e.g. latitude : 10 longitude : 100 ).
To exclude records according to a particular value, use a minus sign (e.g. -maps , -tag : timelines ).
@todo link to JSON query part below.
2.2. The filter builder
The easiest way to create a custom filter is to use the [Filter builder]. You can also access this tool by hovering over ‘Filter builder’ in the left-hand column. In the example, we want to retrieve data about world leaders who are still in office. In this database, a person’s term of office is represented as a ‘Relationship Record’ connecting the person to the country they rule. Therefore this filter should retrieve ‘Relationship Records’.
2.3. Set Filter Criteria
You now need to set one or more filter criteria. In the following example we simply want to find each person who is still in office. For the ‘Criteria’, you should therefore choose ‘End date’ (to look at when people’s terms of office ended) and then choose ‘no data’. If a person’s term of office has no ‘End date/time’, then they must still be in office!
A filter can use several criteria, using the logical operators AND and OR.
With AND : the criteria are cumulative (for any response in the list, criterion 1 and criterion 2 are both true in the same time)
With OR : the criteria are juxtaposed (for any response in the list, either one of the two criteria is true, or both are true.)
The filter builder allows to request in several linked record types : the dropdown menu where choose the filter criteria enables you to navigate to the Record Types linked to (or from) the one in which you are making the query. In the previous example, in a bibliographic database, we are searching only for the dramatic works of a given author. The works are found in the Work Record Type, which contains a field "literary genre", and an "Author" pointer to the Person Record Type: by following the links of the dropdown menu, you can display the fields in the Person Record Type and select the ‘name’ field, for example.
Tips for building your search filter
For complex filters, create smaller elements of the filter, and then combine these to build the full filter.
Using codes (Record ID) in the filter rather than names not only keeps your filters compact, but also ensures that when the filters are saved the codes are invariant, whereas names can be freely changed and it can be a complex task to track these changes and edit all the saved filters.
2.4. Saved filters
2.4.1. Uses of saved filters
Saved filters are the key to setting up useful ‘views’ of your data. Use them to quickly navigate to the records you are working on, to produce sorted lists, to publish sets of data to a web site, to organise the data which have been entered or imported in no particular order. Saved filters not only define a subset of your data and its ordering, but can also set up the way it is presented (e.g. as a map, a formatted report or a visualisation of related records).
If you want to keep your custom filter for later use, you can save it by clicking the [save filter for re-use] icon under the filter search field.
You may wish to save filters that are useful for your analysis, or you may use saved filters to select particular portions of the database to display on the public website.
2.4.2. Accessing Saved Filters
You can access saved filters by hovering over [Saved Filters] to the left of the screen.
3. Build a facet search
Facet searches are a powerful way of drilling down into a database, particularly if they are combined with Rules (there is a rule builder built into the facet search editor) which can pull in related information (such as, for example, spatial information for mapping when the search is based on records which are linked to places but do not themselves contain spatial information). Facet searches allow single and multi selection, alpha versus order by count, effect on speed and optimization of searches with large databases.
You can access the facet builder by hovering over [Facet Builder] at left menu or right clicking on the saved search tree and add Facet search on the bottom of the submenu.
3.1. What is a faceted search?
Faceted searches are interactive tools for searching a database. They are everywhere on the internet. You have probably used one today! Faceted searches allow users of websites like Goodreads, Amazon, the British Library or Google to filter search results according to criteria such as Price, Copyright Status, Rating or Department. Whenever you are allowed to fine-tune search results according to certain criteria, you are using a faceted search.
Heurist allows you to create your own customised faceted searches specifically designed for your database and your users.
To create a new faceted search interface for your database, you can use the facets builder from the Explore Tray.
3.2. When should I use one?
There are two main use cases:
To create research tools for you and your team, so you can easily find relevant records;
To create a public interface for your website.
In either case, the process of building a faceted search is the same. You build the faceted search in the Explore Menu and save it in the Saved Filters tree. If you want to use it for your own internal purposes, you can find it again and re-execute it. If you want to insert it onto a webpage from the Publish Menu, then you can use the 'Saved Filters' widget (see Chapter 9 of this documentation).
3.3. How to build a faceted search ?
3.3.1. First step : general settings
Click on the [Facet builder] item of the left menu : this open a pop-up window in which you can configure the facets.
Here let's assume that we are in a bibliographical database and that you want to search volumes or periodicals recorded in a Record Type named "Manifestation (édition)". Configure your faceted search :
*[Search for (entity type)] : choose the main Record type in which the faceted search will be performed
*[Faceted search name] : the name under which the facet will appear in the saved filters tree once you have saved it.
*[Save in work group] : the folder in which you want to save those facets in saved filters.
*[Display full sets of records] : useful for a website but note that ticking this box may slow down the process if the database is large.
Keep in mind that the record type you choose as input is always the one you will get as output, unless you use the rulesets function (see below).
However, the faceted search allows you to choose your criteria of search from other record types linked to or from this original record type.
Configure the optional features : you can choose
the order in which your results should appear (choose the field of the record type you wish to use to sort the results : by date, by title, etc.)
If you want a box [Simple search] to be display
If you want to apply a preliminary filter to select only a subset in the main record type (for example here : only the volumes and periodicals published in Paris)
If you allow the user to toggle it to expand his or her search to all records.
Configure the display of the faceted search in public interface (in case it is used on a website)
3.3.2. Second step : choosing criteria
The following section of the facet builder allows you to choose the fields you wish to use as criteria of selection.
Note that you can follow the paths to the linked, or linked-from, records (thick the box on top-left of the window).
3.3.3. Third step : configuring the display
The following interface allows you to choose how each facet is displayed :
simple search box : text search
dropdown : display the values of the field in a drop-down menu
list : display the values of the field by the number of their occurrences, one below the other
wrapped : display the values of the field side by side
slider : to select a range of dates
The interface provides other options :
[Show entity hierarchy above facet label] : to be avoided for public websites, but very useful for personal research
*[Accordion view/ Show accordion view] : allows the user to fold/unfold the facets when there are many of them
*[Limit list initially to] : allows you to choose how many responses you want to display when you select the [wrap] and [list] options
*[Rollover] : can be used to write a help text for users
*[Group/Order by counts] : to choose the order of the results' display.
Once the facets are configured, they can be saved. You can re-open it by clicking on it in the saved filters menu, and use it for your own searches and/or to display it on a website.
4. Visualisation panel
The central panel of the interface displays a list of search results. By default, it displays all records in the database sorted by date of creation. It offers several useful features for data management and cleaning, and allows you to perform various operations on multiple records at once.
4.1.[Selected]
This menu provides an access to selection features which apply to the results displayed : [select all], [select none],[show selected],[show as new tab.] To select one or more results, use [ctrl]+click. It also includes additional features :
[Tag] : adding tags to specific records of the database allows users to find it quickly when using a filter is not relevant. Tags can be named and assigned to specific workgroups, or to be user-specific. Once the choosen tags are assigned, you can find it again using a filter (choose for example [any record type], then in [Metadata] : [Tags (terms)])
[Rate] : can be used to assigned ratings to recordings. Note that you can only do this after assigning a bookmark to the record.
[Bookmark]/[Unbookmark]
[Merge]: this function allows to merge two or more records. First select the records, then choose [Merge].
Choose the master record (the one to be kept), then [Merge duplicates].
Choose the fields to be kept in the final merged record, then [commit changes]. Note that the references (i.e. linked records) will be all retained.
[Delete] : delete the selected records from the database.
4.2. [Collect]
This range of functions allows to make by hand personal collections of data : select the records you want to add to a collection, then use [add] (to add it to a collection) and [save as...] to save your selection as a filter.
You can also [remove] records from a collection, [clear all], display the collection in a new tab and/or as a search result.
4.3. [Recode]
This range of functions allows you to make bulk edits to multiple records in the database at the same time. Here you can :
4.3.1. Modify the values of the fields
adding, replacing or deleting the value of a field
add a link to another record ([relate:link])
4.3.2. Modify some aspects of the structure
[Foreign key matching] : This function processes the current query looking for records in another (or the same) entity type, based on matching the values of a field in each entity type. Fields to be matched may be text or numeric. The current query must contain only a single record type (this is enforced to avoid accidental errors). Where a match is found it will insert the ID of the matched record into a record pointer field in the source record.
[Change entity type]
4.3.3. Manage media files
[Local files to remote repository]: to transfer the files to Nakala.
[Remote URLs to local files]: adds a field "File(s) uploaded or remote", whose value is a URL to a remote file.
[Reset thumbnails]
4.3.4. Extract and modify text
[Case conversion]
[Multilined text to html]
[Translation] : translate the value of the selected field. The translation is inserted after the existing value, not in a separate field.
[Extract text from PDF file(s)] : This function extracts text (up to 64,000 characters) from any PDF files attached to a record and places the extracted text in the field specified (by default "Extracted text" (2-652), if defined). Bad characters encountered are ignored. If there is more than one PDF file, the text is placed in repeated values of the field. Text is only extracted if the corresponding value of the field is empty to avoid overwriting any text entered manually.
4.4. [Share] : managing collaborative work
In this section of the menu you will find tools which allow you :
To share a subset of the database with users (see [Notify (email)])
To send emails to the persons registered in the database as records ([Send email]), as far as the records contain - at the least - an email address field. Then choose this (required) field in the first dropdown. For each record selected, one email will be sent to the address stored in this field. If name fields also exist, these can be selected in the next two dropdowns and may be used in the body of the message.
To modify the ownership and visibility of the selected records. Setting the records to ‘public’ status is necessary, in particular, when the website is about to be published.
5. Rulesets
5.1. Why RuleSets?
In a database, important information is often distributed between many different records. For example, imagine you want to know what country a person was born in. In your Heurist database, there may be a 'Person' record for the person, which is linked to a 'Place' record which describes the place they were born. To know what country the person was born in, you would need to locate the 'Place' record for their place of birth, and then see what country that Place is in. In the example below, the Person record for William Shakespeare refers to the Place record for Stratford to describe his Place of Birth:
If you are just looking at one record, you can simply click on the record pointer in the Explore Menu to be taken to the linked record – so really there is no need for any additional tools.
But what if you are examining many records at once? For example, you have filtered the database for a selection of important Persons, and want to see all the Places they were born? Or you have filtered some Places in your database, and want to see all the books published there? Or, more complexly, you have filtered the database to obtain a list of relevant pieces of Legislation, and want to know which Political Party each of the Persons who voted for the Legislation belonged to.
This is where RuleSets come in : you can use a RuleSet to systematically fetch related records from the database, expanding the current result set to include additional relevant records.
Possible applications of RuleSets include:
Researching complex relationships between records in the interface
Fetching additional related records to display on the map or network diagram along with the main records in your result set
Allowing visitors to search for one kind of record (e.g. Educational Institutions) and see another kind of record in the results (e.g. Persons who attended those Institutions)
5.2. How to create a RuleSet ?
To create a RuleSet, hover over [Rules] in the [Advanced] section of the Explore Tray. Choose a workgroup to save the RuleSet under and click 'add'. Or click right on the RuleSet tree and select [New RuleSet]. In the RuleSet editor, you can step from one Record Type to another using Record Pointer and Relationship Fields. It is possible to step in two directions. In the above example, you could step from the Person Record for Shakespeare to the Place Record for Stratford, or you could step from Stratford to Shakespeare. At each step, you can optionally apply a filter, which you can define using Heurist's Filter Builder. You can also add a RuleSet to a predefined filter.
5.2.1. Building a RuleSet : an example
In the image above, the Ruleset looks at all the Persons in the current results set, and finds the Places where they died. It then finds any Life Events associated with those Places. Thus, if you filter the dataset to find some interesting people, you could answer the question: What Life Events are recorded for these Persons' places of death?
As an added element, the Places can be filtered when the RuleSet is applied. To add a filter, either type the filter directly into the box using Heurist's query language, or click the pencil icon to use the Filter Builder. In the screenshot, Places are filtered so that only Islands will be considered. Thus the question becomes more specific: What Life Events are recorded for the Islands on which these Persons died?
If you click [Add new Rule], then you can include a second, separate set of steps to fetch related records. For example, if you wanted to see the Places of Birth as well as the Places of Death for the Persons in the result set, then you would need to add a new rule to the RuleSet.
5.2.2. Integrating RuleSets with other tools
Once you have saved a RuleSet, you can integrate it with other tools in Heurist. For example, if you have defined a faceted search that queries the Borrowing Records in a Library database, you could then apply a RuleSet to replace all the Borrowing Records in the results with the Persons who actually borrowed the books.
The main places you can apply a RuleSet are :
To the results of a Faceted Search
To the results of a filter created using Heurist's Filter Builder
But more generally you can apply a ruleset to any set of results and if it is appropriate it will expand the set of results folowing the rules defined.
6. Advanced Users: Introduction to JSON Queries
A Json query is an array of objects (predicates). Note that this JSon format is generated by the rules+filter button
Each predicate is a pair: {“keyword”:”value”}:
A keyword stands for record header field, detail or link predicate.
The value depends on the keyword. It may be literal, csv. It may be preceded by a compare operator or contain a range or % operator.
example :
{"q":"sortby:-m after:"1 week ago""}
For link predicate, the value is a sub query (another set of predicates).
example :
{"q":"sortby:-m after:"1 week ago"","rules":[{"query":"t:12 relatedfrom:14-4533 ","codes":"14","99","4533","12","",4],"levels":[]}]}
Heurist queries, in both JSon and simple filter forms, can be used in several contexts. The table below outlines the various contexts in which queries can be used, and explains the considations that must be taken into account in each context. In some contexts, the query must be placed within another JSon object whose name is "q:" and whose value is the desired query; this is called the "q:" parameter.
Context
JSon or Simple Filter
"q:" parameter
Example
Main page search box
BOTH
No
sortby:-m after:"1 week ago"
CSV output query
Simple ONLY
No
f:149:34
Mappable query
JSon ALWAYS permitted.Simple Filter permitted ONLY IF no rules are applied to the query
Yes
{"q":"sortby:-m after:"1 week ago"","rules":[{"query":"t:12 relatedfrom:14-4533 ","codes":"14","99","4533","12","",4],"levels":[]}]}
Facet search pre-query
BOTH
No
{"f:10":"1914-12-31T23:59:59.999Z<>1931-01-01"}
Expansion rules
JSon ONLY
No
[{"query":"t:12 linkedfrom:16-90 ","codes":["16","90","","12","",2],"levels":[]}]
Specifying database for mappable query data sources
The user can specify db parameter in query field of “Mappable query datasource so that it can be rendered from any database.
For example:
{"q":"t:12 f:26:108","db":"osmak_38"}
6.1. Syntax
6.1.1. Simple filter syntax
Up to version 4, Heurist used a simple search-engine style of filter syntax which is documented in the help link next to the filter field on the Explore page. This syntax is still supported and is useful to quickly find things eg. by simple text searching.
In versions 4 and above, the syntax of the Heurist queries is based on JSON (JavaScript Object Notation), which allows for programmer-writable inputs. Click here ==missing link== for a basic introduction to JSON syntax. For mere human beings, the Filter Builder will build the JSon queries, which may then be edited by hand for small changes.
Queries are written as JSON objects, which begin and end with braces
{}. An object contains zero or more name-value pairs, in which the name and values are separated by a colon
: Multiple values are separated by commas. All strings and comma-separated sequences must be enclosed in double quotation marks for the query to be valid in JSon syntax.
Many basic Heurist queries can be performed using a simplified version of JSON syntax in which the braces and double quotation marks are removed.
For example, a basic query is written in simplified syntax as
f:1:a and will return the set of all records whose titles contain the letter "a".
Negation is expressed in simple syntax by placing the minus sign before the whole object:
-f:1:a returns all records whose titles do not contain the letter "a".
Be cautious when using simplified syntax, as not all queries can validly be expressed in this form. For example, the "==" operator is not implemented in simplified syntax, only in formal JSon syntax. If in doubt or if an unexpected result set is returned when using simplified syntax, please revert to using full formal syntax as described below.
6.1.2. JSon query syntax
In a Heurist query, the name represents the field or attribute being matched, and the value represents the logical predicate that is matched to it. For example, the query
{"t":"1"} is interpreted as follows: "return all records such that their attribute "t" (record type) matches the value "1" (relationship record)". Thus this query returns all relationship records in the database.
It is possible to write a name as an object, which is used to specify the field being matched. For example, the object {f:1} (a simple filter object interpreted as "the field with code 1") denotes the Name field of any record, so
{"f:1":"a"} is interpreted as follows: "return all records such that their field with code 1 (i.e. their Name field) contains value "a"". This query returns all records whose titles contain the letter "a".
To include multiple query terms using the JSON syntax, you need to enclose your query in square brackets: [ ]. For example, to search for all Persons in the database whose surname is "Patel", you could type:
[{"f:1":"Patel"}, {"t":"Person"}]
Basic query terms
The following table gives the names and values that constitute basic queries, with an explanation of their meaning and use, as well as examples in both simple filter format and formal JSon format. A result set from a simple filter search is automatically sorted, while a result set from a JSon search is unsorted by default.
Name (Meaning)
Value
Result
Simple filter (sorted by default)
JSon syntax (unsorted by default)
t (record type)
number OR string
Returns all records of type value. If value is a number, it refers to the index of that record type, and if value is a string, it refers to the name of that record type
t:1 returns all Relationship Records;
t:Person returns all Person Records
{"t":"1"} returns all Relationship Records;
{"t":"Person"} returns all Person Records
f:#, field:# (field type)
string
Returns all records whose field with index # contains value. Hot tip: The field number is optional. If you wish to search all the fields associated with the records, then you can simply use "f".
f:1:a returns all Records whose field #1 (Title) contains "a".
f:a returns all Records which have an "a" in any field
{"f:1":"a"}returns all Records whose field #1 (Title) contains "a";
{"f":"a"}returns all Records which have an "a" in any field.
ids (record ID)
number
Returns all records with record IDs value.
Separate multiple IDs with commas:
ids:51,52,53returns Records #51, #52, #53, #54 in the database
{"ids":"51,52,53,54"} returns Records #51, #52, #53, #54 in the database
linkedto(linked records)
number
Returns all records that point to the record with ID value.
linkedto:123 returns all records that point to Record #123
{"linkedto":"123"} returns all records that point to Record #123
linkedfrom (linking records)
number
Returns all records that the record with ID value points to.
linkedfrom:123 returns all records that Record #123 points to
{"linkedfrom":"123"}returns all records that Record #123 points to
related (related records)
number
Returns all records that have a relationship to the record with ID value.
relatedto:123 returns all records related to Record #123
{"relatedto":"123"} returns all records related to Record #123
Extending queries
It is possible to extend the value of a query using commas. Thus in formal JSon syntax:
{“f:1,4”:”find me”} returns all records in which either field #1 or field #4 contains the string "find me".
It is also possible to include relational operators in the value, in order to specify the match more precisely. For example
{"f:210":"==Poet"} in which the relational operator = requires an exact match, while the relational operator == gives a case sensitive match. Please note that the PHP operator "===" (identity) is not implemented in Heurist.
Special attribute queries
The following queries target special attributes of records such as Ownership, Visibility, Date Modified, etc. These must be written in formal JSON syntax, for they do not work with simplified syntax.
{"addedby":"29,1000"}
{"addedby":"-osmakov"}
{“owner":1}
{"owner":"Database Managers"}
{"access":"hidden"}
{"access":"viewable"}
{"access":"public"}
{"access":"-public"}
These can be combined:
{"owner":3,"access":"viewable"}
In order to query multiple types of record whose visibility is not public, use the following query in simple filter syntax :
visibility:-public (t:24 or t:11 or t:25 or t:27 or t:28 or t:29 or t:44)
or the equivalent query in JSon syntax:
{"access":"-public","t":"24,11,25,27,28,29,44"}
6.2. Logic
These are the logical keywords used in queries:
not
all
any
OR
AND (default)
notall
NOT ( AND )
notany
example :
notany:[{"title","Black"},{"title","White"}] => NOT ((rec_Title = ‘Black’) OR (rec_Title = ‘White’))
NOT ( OR )
By default the set of predicates conjoined by AND:
[{"title":"President"},{"f:1","Nixon"}] stands for
(rec_Title = ‘President’) AND (dty_ID=1 and dty_Value=’Nixon’)
To shorten the query, it is possible to unite predicates of one level into single object:
{"title":"President", "f:1","Nixon"}
It is also possible to nest logical conjunctions. For example:
{"not":{"any":[{"title":"Milano"},{"title":"Veneto"}]}} returns every record whose Title mask does not contain "Milano" or "Veneto".
6.3. Keywords
6.3.1 Record headers
f, field, **Example:
[{"t":"10"},{"f:1":"goethe"}]
url,u, rec_URL
title, rec_Title
addedby, rec_AddedByUGrpID (takes as a value an user or users group ID)
Example :
{"q":"addedby: 7"}
added, rec_Added
Example:
{"q":"added: 2025-07-02"}
date, modified
Example :
{"q":"modified: 2025-07-02"}
after, since, before : Synonyms for modified with compare operator in value
workgroup,wg,owner,rec_OwnerUGrpID
id, ids, rec_ID
t, type, rec_RecTypeID (takes as a value an ID or a string)
Example:
{"q":"t:109"} ou
{"q":"t:Place"}
latitude, lat, longitude, long, lng
Links
linkedto
Example:
[{"t":"102"},{"linked_to:1158":[{"t":"103"},{"title":"goethe"}]}] (is for : Which records of the Record type with ID 102 point to records of the Record type with ID 103, whose title includes the character string "goethe"?) Find records which have linked records specified in value for this predicate (subquery or csv of ids). Resource field ID (:x) is optional
linkedfrom Find records that are linked from records. Resource field ID (:x) is optional
relatedto Find records that relates to records from subquery. Relation type (:x) is optional
relatedfrom Find records that relates FROM records from subquery Relation type (:x) is optional
Example:
[{"t":"10"},{"relatedfrom:1103":[{"t":"102"},{"f:1":"BAVIERE"}]},{"sortby":"t"}]
links @todo ==to verify==
Bookmarks, Tags
user, usr,bookmarked by user
tag, keyword, kwd
Example :
[{"kwd":"à corriger"},{"sortby":"t"}]
6.3.2. Values for Keywords
Literal:
"f:1":"Peter%"
CSV:
"ids":"1,2,3,4"
WKT :
"f:5":"POLYGON ((30 10, 40 40, 20 40, 10 20, 30 10))"
6.3.3. Possible Operators Within Keywords
"X<>Y" : turns into BETWEEN X AND Y
"-X" : NOT ( )
"=X" : suppress LIKE operator for freetext field type
"X” : applicable for numeric and date values only
A vérifier/recontextualiser
@todo
notes, n Synonym for f:[DT_SHORT_SUMMARY] Where DT_SHORT_SUMMARY is replaced with local code of concept 2-3 todo
1931-01-01"} Expansion rules YES Generated by expansion rule wizard This is a part of the full mappable query JSon opject No [{"query":"t:12 linkedfrom:16-90 ","codes":["16","90","","12","",2],"levels":[]}]
Ch 08 : Result sets, manipulation, custom reports and visualisation
Ch 8: Result Views and Export
Documentation written on 04/11/2025 by Sylvain Besson (MSH Lyon Saint-Étienne / CNRS)
Updating 25/06/2026 by Vincent Paillusson (HTL)
1. Record View
Explore → Filter → Record
The Record view shows all the elements of a recording. The differents fields and associated metadata and possibly relationship between recordings.
How to start:
To begin, click on [Explore]. ①
Perform a search or use a [Saved Filter] that returns the records you want to include in your report. ①
Select a record ②
Click on [Record] ③
Once on the [Record] view selected, the record’s metadatas appear. There is several informations:
Title ①: title of the record
Icon ②: record’s type icon
Record H-ID ②: Intern Heurist record ID
Workflow stage ④ which give some information on the workflow stage
Media ⑤: Picture of the media
Focus on medias
If a media exists in the record, a thumbnail is displayed
It is possible to display in [full screen]
① or viewing it in popup
②
The media can also be displayed on [Mirador] (using Heurist’s automatic IIIF manifest) ③ or [OpenSeadragon]
viewer ④
You can [download]
it ! ⑤
By hovering over [description], a description appears if this field has been completed ⑥
Finally by going over [rights], the rights of the media appears if it was filled during the record creation ⑦
Click on [More…]
Cite as ① : The record can be cited in XML or HTML. The updating date of the record is shown as the lastest modification date.
Added ②: Creation date of the record
Updated ③: Last record update
Ownership ④: Communicate who owns the record and who can read it
Rating ⑤: You can rate each record from one to five
Tags ⑥: You can tag records to find them more easily
2. List view
Explore → List View
[List view] allows essentialy to show the whole selected data in table format ③.
You can choose the field you need ①. You can also save the settings ②. It is also possible to export by simple copy/past (CSV with tab as separator), by excel format and PDF format ③.
🛟
Tip: If you want to export data with more export formats, you may want to try the Export view described in the following section.
3. Export View
The Export view allows to export the request results under differents data formats:
CSV ①
XML ②
JSON ③
RDF ④
GeoJSON ⑤
KML ⑥
GEPHI ⑦
IIIF ⑧
HuNI ⑨
Several formats can be exported as a data feed, as well as a fixed format file. The data feed capability is particualrly useful for sending live data to a processing workflow, often combined with a saved search which filters the required set of output records. It can be a very useful, simpler alteraitv eto usign th Heurist API.
CSV / TSV (Delimited) files
When you click on [CSV], a pop-up opens allowing you to choose the fields you want to export and a range of settings.
Note that by default CSV files are exported as tab-separated, since this causes much fewer problems with complex text (which often contains commas and unmatched quote marks, but very rarely contains actual tab characters)
CSV/TSV exports are a particularly good way of temporarily exporting some fields, carrying out some manipulation in an externa program eg. a spreadsheet, Open Refine or R, and reimporting the results with Populate > Import - Delimtied text / CSV. Because all exported CSV/TSV files automatically include the Hursit ID (H-ID)_ which is uniqiue to each record, it is very easy to reimport the data into the soure records, overwriting or adding to existing values in the same or differnet fields.
First, you must choose the records you want to export ①. You can choose between the current result set and any single type of record occuring within the resultset.
We STRONGLY recommend only exporting one record type at a time. Delimited files are really not meant for dealing with heterogeneous data, and mixed exports will restrict the exportable fields to record metadata and shared fields.
After the records selection, you choose one of the two available export settings ② :
- a single joined file
- a file by record type (if multiple types are selected).
You can choose to display the fields in either Form order (the default) or alphabetic order. Alphabetic order may make it easier to select fields in some circumstances.
The main step of the export setting is to select the fields you want to export ③ . They can be any type of field including the constructed title, record pointers and relationship markers.
You can export metadata about the records ④ as well as the data from the data fields ⑤ .
If you have a current resultset with more than one record type in it, you will only be able to choose the metadata and fields which are shared by all the record types in the resultset.
If the record type you are exporting contains record pointer or relationship marker fields, you can drill down into those record types and export fields from within those records. They can be included in the main file ("single joined file") or exported as separate files with linking IDs ("File by record type").
You should use "File by record type" where there are multiple values in record pointer fields, since the values in each related record will need to be kept separate. For simple cases without repeated value record pointers "single joined file" may be appropriate.
Note how you have some additional options appearing on the right against any selected fields.
For most purposes you will want to use Value, as the other options can be done more effectively in an external spreadsheet. However Group by will group records into a single record for each value encountered in the column, while Count will count the number of occurrences for the field in those groups. The results can be a little hard to interpret.
🛟 Tip:
The H-ID is ALWAYS included in the export because it uniquely identifies every record, so it is essential if you need to import/update data back into the database or to make links between records eg. record pointers
Finally, when ending the CSV export setting, you can change the field/column delimiter for the csv (for example: tab, semicolon, comma, etc.) and character used for quotemarking the textual content.
You can also save settings ⑦ by giving it a name in order to use it later.
Handling of record pointers and relationships
For all other export types, it is possible to choose between:
Export the records and their relationships (relationship of the type pointer or the relationships maker)
Export only the pointer relationships
Don’t follow the pointer relationships or the relationships makers
Follow all relationships including inverse pointers (🚨warnings: it could export the entire database)
XML
“Extensible Markup Language (XML) is a markup language and file format for storing, transmitting, and reconstructing data. It defines a set of rules for encoding documents in a format that is both human-readable and machine-readable.” (source: Wikipedia)
Heurist defines an XML schema called Heurist Markup Language (HML). HML can be used both as an interpretable archivable format (it is included as the primary element of Publish > Safeguard file) and as a data source which can be transformed with XSLT transforms, Python, PHP, or many other languages to a required format.
Check the box "Include human-readable names and local IDs for everything" if you plan to look at the XML file and interpret its structure (this will create a very large file duer to repetition). It is often better to export an explanation of the structure through Populate >Heurist XML/JSON - Download template.
JSON
“JSON (JavaScript Object Notation) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of name–value pairs and arrays (or other serializable values).” (source: Wikipedia)
The choices are similar to XML/HML, except that it cannot include the human readable forms. However, as with XML, you can download the information in the form of a JSon tempalte through Populate >Heurist XML/JSON - Download template.
RDF
“The Resource Description Framework (RDF) is a method to describe and exchange graph data.” (source: Wikipedia)
In this format, you can specify the serialisation you want: rdfxml, json, ntriples or turtle
As this function is still in development (July 2026) it requires a special password to access. Contact the Heurist development team for further information.
GeoJSON
“GeoJSON is an open standard format designed for representing simple geographical features, along with their non-spatial attributes. It is based on the JSON format.” (source: Wikipedia)
You can select the export detail between: No = no detail, Inline = Inline detail, and Full = maximum detail
KML
“Keyhole Markup Language (KML) is an XML notation for expressing geographic annotation and visualization within two-dimensional maps and three-dimensional Earth browsers. It is best known for its use in Google Maps but is widely importable into GIS and mapping packages” (source: Wikipedia)
There are no options for this export format, it is exported immediately as soon as you click on the button.
GEPHI
GEPHI export generates a GEFX file which can be loaded immediately into GEPHI. Note that this export function is also available directly within the network visualisation graph in the Network tab.
In addition to the normal node and edge fields, you can choose to add additional fields to the export ①.
This leads to the pop-up below. We recommend only selecting fields relevant to the record type being exported. You can also export with jsut the default fields ② which will be sufficient in most cases.
🛟 Tip:
It can be useful to check the box limiting the export to the first 1000 nodes in order to check that the export gives you what you want in GEPHI, befor exporting a very large dataset.
IIIF
“The International Image Interoperability Framework (IIIF, spoken as ‘triple-I-F’) defines several application programming interfaces that provide a standardised method of describing and delivering images over the web, as well as “presentation based metadata”[1] (that is, structural metadata) about structured sequences of images” . (source: Wikipedia). IIIF can also handle tiled image delivery andimage annotation.
Heursit acts as both a IIIF manifest delivery system and image server, and as an IIIF display and annotation client, notably through the use of Mirador Vsn 4 and Open Sea Dragon viewers, and the MAE annotation framework. Heurist can also read and atomise manifests containing annotations, and recompose manifests including those annotaitons and others created within Heurist.
IIIF is a rich and complex system. Heurist's IIIF implementation is discussed in detial in chapter 8e.
HuNI
“HuNI (pronounced “honey”) brings together information about the people, works, events, organisations and places that form Australia and Canada’s past and present.” (source: huni.net). It is an old infrastrucute project dating to the 2000s with limited functionality (essentially harvesting simple metadata from 40+ Australian sources, providing simple search, bookmarking as a 'collecction' and exporting a CSV file with title and URL of the bookmarked records. The HuNI export format has the particularity of exporting one XML file per record. It may be of some use if that fits with your needs.
HTML
This option exports HTML pages for public records (one file-per-record) using the Record view format.
📊 Export‑type quick‑reference table – Which format to choose?
Need
CSV
XML
RDF
JSON
GeoJSON
KML
Gephi
IIIF
HuNI
Spreadsheet
✅
✅
Markup (tagging)
✅
✅
✅
✅
✅
✅
✅
Versatile (generic data)
✅
✅
✅
✅
✅
✅
✅
✅
Spatial (geographic)
✅
✅
Networks
✅
High‑resolution images
✅
1Password menu is available. Press down arrow to select.
8a: Getting started with custom reports
Documentation rédigée le 06/11/2025 par Shannon Bruderer mise à jour le 05/12/2025 par Shannon Bruderer
What is a Custom Report ? :
A Custom Report is a template that structures your database records into various output formats such as HTML (for web display), plain text (for transfers without formatting), CSV (for spreadsheet work, e.g. in Excel or Open Office), JSON (for data feeds), or XML (for tagged data exchange).
Note however that CSV, JSON and XML are handled much more easily in the Export tab in Explore unless some very specialised formatting is required.
The development of custom reports can be quite a slow process, so it is best to plan well what reports you need and apply good naming conventions. Some level of simple HTML will be required, and knowledge of CSS will allow for much greater control of the output. PHP and JS functions can also be included (optional).
Custom Reports are useful when you need to extract and format specific data for further analysis or publication. They are particualrly useful in formatting data to appear in web pages (see chapter 9). They can also be used to download formatted information or set up feeds of data for other purposes.
They are also useful for displaying a single record in Record View, a popup on the map, or wherever data needs to be displayed in response to selection of one or more records.
Tip : Before creating your Custom Report, define clearly what sort of display you plan to do. Your formatting goal will determine both the data you select and the way you format the output.
Reports are built using Smarty, a templating language that combines standard HTML with Smarty tags to dynamically insert data from Heurist.
You may see this message above your report format in the middle panel. CSS and JS are disabled by default in report formats and websites generated by a database as a security precaution, to avoid unwanted actions from parasitic databases. If you wish to use these - many websites will want to customise their appearance or behaviours more than the default settings allow - you should contact the server adminstrator who will add the database to a list of authorised databases.
If you change the name of a database, clone it or create a new database it will not allow CSS or JS.
The new name will need to be added to the list.
How to start :
In the Explore menu ① you can focus on a specific record type (Explore > Entities) and select the record type from the list ②
You may wish to perform a more flexible filter, either by entering it in the Filter field ③ or using the Filter builder or Facets builder just below it, or use a [Saved Filter] from the Filters ection ② that returns the records you want to include in your report.
The selected records will appear in the middle section of your screen. ④
To work with Custom Reports, click on the [Report] tab ⑤
The Custom Report template for your filtered data will appear below the [Report] button.
The report shown in the backend interface is limited to 50 records by default (can be increaed to 200 or 500 in Design > My Preferences) and ends with a line stating this limit, as it is really designed as a preview function. To view the full results click the globe or download icons - see below).
The Toolbar
In the upper part of the Custom Report tab, you’ll see a toolbar:
Edit Tool
Click [Edit] to open the template editor and start writing your Custom Report with Smarty.
The editor is split into three panes ①, ⑤ and ⑨
:::info Tips :
Actions pane ① :
Insert fields, loops, and conditions via dropdown helpers ②, and a tree view ④ selected with the record types dropdown ③
The tree view ④ allows you to select multiple fields, including from the metadata attached to each record (ID, date of addition, owner etc.), the method of representation of term fields (label, description, code), and to drill down into connected records through record pointer fields or relationship markers.
The fields can be organised in record form order (default view, showing the tabs and headings in bold font), or in alphabetic order. The tree can also show record types which have record pointers pointing TO the record type selected (Show linked-from record types checkbox)
Having selected one or more fields, make sure the cursor is positioned where you want them inserted in the report format and click Add selected fields. This will display the following popup;
By selecting the checkboxes you can accompany insertion with various additional functions and pieces of information - the two selected by default will be useful in many cases to get some sort of reasonable default output.
Fields can be inserted one-by one (Insert field) allowing the accompanying information to be changed between fields, or you can insert all the remaining fields with the same accompanying information by clicking Insert all. Fields can be omitted with Skip.
The pattern insertion dropdown ② above the treeview function allows insertion of a number of simple html patterns such as tables and record links, as well as patterns to carry out some action eg. writing some label or separator, at the beginning or end of a loop. Note that the list of available patterns may change as new capabilities are added.
Editor pane ⑤ : write and edit your HTML + Smarty template here.
In the editor, you’ll see the template you selected or the default starter message/template (note that this template may change through time as we improve on it, but it will contain a basic loop for records and some instructions at the end to help you get started).
{* This is a simple Smarty report template which you can edit into something more sophisticated.
It should give basic output for any database, as it uses the standard record types which are part of all databases.
Enter html for web pages or other text format. Use tree on the left to insert fields, loops and tests.
Use this format to include comments in your file, use for output of html comments.
Smarty help describes many functions you can apply, loop counting/summing, custom functions etc. *}
For Smarty syntax please see the following chapter (8b).
Preview pane ⑨ : shows the output when you click [Test] ⑧ .
You can choose to truncate the preview to n records ⑥ and select how to handle debug messages, warnings, and errors ⑦.
Click [Test] to preview ! Nothing is saved when testing.
Use [Save] (or [Save As]) to store your template and keep versions.
Use Ctrl+Z / Cmd+Z to undo recent edits. You can undo a lot of edits by repeating this.
We strongly recommend using the test function frequently making only one or two changes at a time and clicking Test to see the results. If somethign doesn't work, you can immediately undo it and try an alternative. Undo can be applied repeatedly.
Don't get tempted to write a lot of code and then test it because then you will have trouble finding the problem.
Rename
Allows renaming of an existing template. Note that if a template is renamed, any URL or scheduled regeneration which uses the old name will fail.
Create a new template
[Create a new template] works similarly to the [Edit] tool. It opens the same editor interface where you can create a new Custom Report template from scratch.
Use it when you want to start a fresh layout instead of editing an existing one.
For Smarty syntax please see the following chapter (8b).
Delete the Selected Template
The [Delete] tool allows you to delete the currently selected template.
When clicked, a warning message will pop up asking for confirmation.
It will display the name of your template ① like name_file_.tpl . As here for exemple "Basic (inital record types).tpl"
Click [Proceed] to confirm deletion, or [Cancel] to abort the action.
Import and Export Templates
The Import ← and Export → tools allow you to share and reuse Custom Report templates.
For this we have developed a 'global template' format (.gpl) which uses Heurist's unique Concept IDs so that the template can be usd by any database that includes those concepts (definitions of record types, fields and terms). Template files stored in the Heurist database are the same as global files except that they use local codes rather than the unique global concept IDs.
Templates can only be exported from a registered database to ensure that there are Concept IDs for any definitions used in the template. If the database is not registered you will see the following message.
Import lets you upload an existing global template file (.gpl) and convert it to a local template file (.tpl)
Export lets you download your customized template as a .gpl file, so you can back it up or share it with others.
Obtain the URL, JavaScript to embed a report, and set a publishing schedules
The [Publish] option lets you :
embed a Custom Report in an external website in another CMS
schedule periodic regeneration with caching for faster load times on large/complex reports.
Embedding
The dialogue above gives an iframe instruction to embed the report into another website. Switching to javascript wrap will give an alternative text such as:
The Open in new window link is a useful way of seeing the report cleanly and for obtaining a URL for use elsewhere.
The Content-type dropdown allows a number of differnt output formats to be specified, setting a parameter on the URL used by Open in new window. html and text are the two most useful, the others produce generic outputs which may or may not be of any use. To obtain text output, do not include any html tags in the report format.
Setting up a scheduled (cached) report
Pros/cons of scheduling
Much faster for large tables, complex calculations, or media-heavy pages.
Content is a snapshot at the last generation time (not strictly real-time), so frequency of update needs to be approriately set
The first screen shows any existing scheduled actions:
Adding a new report schedule pops up a dialogue to define the parameters of a new schedule using the current filter ("Query") with a number of different options - a title to identify it, the report template to be used, the frequeny with which to regenerate the output (the default value of 1440 minutes = daily, 0 = only manual regeneration, which is useful for data that will never, or very rarely, change).
Download
This allows the download of a plain text file without html formatting (assuming you did not use html tags in the report format)
Print
The [Print] buttom simply generates a PDF of the output from your current Custom Report template. It’s a quick and convenient way to export information in an easy readable and shareable format.
Tip: Don’t hesitate to use this feature to:
Enrich your Data Management Plan (DMP),
Place in a hardcopy archive,
Keep track of specific datasets, or
Share information with colleagues who may not be comfortable navigating Heurist or other “sophisticated” data formats.
Refresh
Click on the [Refresh] buttom to update the data used by your Custom Report template. For filters other than Facet filters (where you must make surther selections) you may also simply hit the Filter button to rerun the filter, which will cause the report to be rewritten.
If your database has been modified (new records, edits, deletions) but the output of your report does not reflect these changes, simply hit Refresh to reload the most recent data and ensure your preview is accurate.
8b: Custom reports - Advanced functions
Advanced topics in custom reports
Note (July 2026): the content was copied via markdown export and lost much of its minor formatting. The images in particular have been downgraded. The original source is here: https://docs.google.com/document/d/1Jyytaln1-aCm3paZ4rBKho0puXBGaJ97/edit
This chapter contains lots of undigested tips for advanced users, skip the first 10 pages or so to get to this material.
Smarty Syntax
If you are not familiar with Smarty or the Smarty syntax, the Smarty Site has a range of information and resources on using the Smarty Report Template Engine, including complete Smarty documentation.
This topic provides an introduction to some basic syntax elements when you are using the Actions Pane to create simple reports.
Advanced features
SMARTY provides a range of features that can improve your reports. For a full explanation, visit the SMARTY documentation.
Template plugins
Template plugins provide advanced template functionality. Template plugins include:
Functions
Block Functions
Modifiers
Plugins are always loaded on demand. Only the specific modifiers, functions, resources, etc. invoked in the templates scripts will be loaded. Moreover, each plugin is loaded only once, even if you have several different instances of Smarty running within the same request.
Main records
The foreach statements enclose a loop which outputs information for each record in the query result. Fields can be inserted with the insert links next to each field. Use the if links to insert tests based on the value of a field (e.g.. to only output text if a field is set).
Subrecords
Further loops can be inserted to output multiple sub-records within the main record loop, using the loop link after the subrecord name. Fields within sub records can be inserted with either the in or out links; use the in link to insert a field within a loop, use the out link to insert a field outside a loop.
Comments
Syntax: {* This is a comment *}
Comments are useful for making internal notes in your template. They are completely ignored in your template file and are invisible to public view (unlike ).
Variables
Synatx: $foo
Variables allow you to dynamically replace the variable by data when the web page is created. For example, instead of writing the record title in the template, you can use a tag like {$title} in place of the title.
Variables can contain numbers, letters and underscores.
You can apply maths to variables that contain numbers. For example:
{$foo+1}
{$foo*$bar}
{$foo->bar-$bar[1]*$baz->foo->bar()-3*7}
Smarty has several different types of variables. The type of the variable depends on what symbol it is prefixed or enclosed within.
Variables in Smarty can be either displayed directly or used as arguments for functions, attributes and modifiers, inside conditional expressions, etc. To print a variable, simply enclose it in the delimiters so that it is the only thing contained between them.
Arrays in Smarty reports
You may use arrays in smarty report easily. Access element by its index First element: {$newValue[0]}
Or use standard array functions. Print array: {print\_r($newValue,true)}\
Implode array: {implode('\*', $newValue)}\
Functions
Smarty has many built in functions for formatting, sorting, totalling etc.
You can also use PHP functions (built-in or ones you define in the code) in a Smarty template, for example:
{$r,fldname} will output the value of the field
{$r.fldname|upper} will output the value of the field converted to upper case (Smarty function)
{str_pad($r.fldname},10,"0", STR_PAD_LEFT) } will pad the string to length 10 with leading zeroes (PHP function)
Every Smarty tag either prints a variable or invokes some sort of function. These are processed and displayed by enclosing the function and its attributes within delimiters like so: {funcname attr1="val1" attr2="val2"}.
Smarty allows for:
built-in functions. For example, {if}, {section} and {strip}. There should be no need to change or modify them.
customer functions. These are additional functions implemented by you via plugins. They can be modified to your liking, or you can create new ones.
Built in functions include:
{assign}
This is used for assigning template variables during the execution of a template.
{assign var="name" value="Bob"}
{assign "name" "Bob"} {* short-hand *}
The value of $name is {$name}.
The above example will output:
The value of $name is Bob.
{$var=...}
This is a short-hand version of the {assign} function. For example:
{$name='Bob'}
The value of $name is {$name}.
The above example will output:
The value of $name is Bob.
{for}
The {for}{forelse} tag is used to create simple loops. The following different formats are supported:
{for $var=$start to $end} simple loop with step size of 1.
{for $var=$start to $end step $step} loop with individual step size.
{forelse} is executed when the loop is not iterated.
For example:
or
{/if}
V2 - > r in second for loop can be used in the same way as in any other for loops of records
$repeatsorted=array()}
{foreach $repeat as $item name=valueloop}{* *}
{$item=$heurist->getRecord($item)}
{$repeatsorted[$item.recID] = $item.recTitle}
{/foreach}
{capture}{asort($repeatsorted)}{/capture}
{foreach $repeatsorted as $itemsorted name=valueloop}{* *}
{$r=$heurist->getRecord($itemsorted@key)}
{/foreach
***To get info from the relationship record ***
(I've added this to z_Ian_Text report):
Use h6-alpha
{* Get infromation from the relationship record *}
{$rel_record = $heurist->getRecord($Relationship.recRelationID)}
{$src_info = $heurist->getRecord($rel_record.f1160)}
Source de l'Information: {$src_info.recTitle}
Start Date: {$rel_record.f10}
End Date: {$rel_record.f11}
I have put in rubbish dates 1111 and 9999
getRelatedRecords returns an array of related records with additional header fields: recRelationType, recRelationNotes, recRelationStartDate, recRelationEndDate.
Note that when using getRelatedRecords and getLinkedRecords, it is not possible to detect what relationship marker field generated the particular relationship. We don't keep this info. You may filter out the required record by rectype and relation type
Detecting if a linked record is visible to public
Detection, in a custom report, whether a linked record is visible to the public
**{foreach $r.f1107s as $f1107 name=valueloop}{\* Other sources \*}**
**{$source=$heurist-\>getRecord($f1107)}**
**{if ($source.recNonOwnerVisibility=='public')}**
or: {if ($source.recIsVisible!==false)} {* Hide non-public related sources if not logged in *}
Advanced HTML, CSS and JavaScript
To unlock the full power of the Custom Report tool, you need to have CSS and JavaScript enabled for your database. To do this, please contact your server administrator. On the Sydney and Huma-Num servers, the administrator is ian.johnson@sydney.edu.au . If you are using a hosted version of Heurist at another institution, you will need to inquire there about who has adminstrative rights.
Along with HTML, JavaScript and CSS are the building blocks of the web. JavaScript is a fully-featured programming language with inbuilt tools for interacting with web pages through the Document Object Model (the DOM). CSS (or 'Cascading Style Sheets') is a simple formatting language that allows you to describe how you would like different page elements to be formatted.
There are many ways you can embed JavaScript and CSS into a Heurist website. For a full discussion, please visit the JavaScript and CSS pages of this help system.
Once you have enabled JavaScript and CSS, you can structure your reports using more advanced features.
Note that the custom JS and CSS defined at the site or page level (in CMS Home or CMS Menu_page) are not applied to custom reports. You must add the needed CSS and JS code directly in the report.
Note that you will not be able to use php code and functions within smarty on Heurist. Most php functions have been restricted for security issue. Please contact the Heurist development team should you need to use php code.
Using Advanced HTML Elements
By default, you are able to use the following tags to structure your Custom Report:
-
Or use standard array functions. Print array: {print\_r($newValue,true)}\
-
{for $foo=1 to 3}
- {$foo} {/for}
- 1
- 2
- 3
-
{for $foo=3 to $to max=3}
- {$foo} {/for}
- 3
- 4
- 5
-
{for $foo=$start to $to}
- {$foo} {forelse} no iteration {/for}
Book Title: {$r.f1|capitalize}
{* Data in the database: 'the history of Tom Jones, a Foundling. In four volumes.' *} Book Title: The History Of Tom Jones, A Foundling. In Four Volumes As you can see, to apply a modifier, simply type the pipe "|" character after the data, and then type the name of the modifier you wish to use. It is possible to use multiple modifiers at once, and also to change their behaviour. For example, your Books database may contain many long titles, as well as many titles that are not capitalised correctly. You can easily shorten ('truncate') the tiles as well as capitalising them like so:Book Title: {$r.f1|capitalize|truncate:25}
{* Data in the database: 'the history of Tom Jones, a Foundling. In four volumes.' *} Book Title: The History Of Tom Jones,... As you can see, to use another modifier, you can simply type another pipe "|", and put the name of the next modifier after it. If the modifier needs you to specify some settings, you can do this with a colon ":". In this case, you can tell 'truncate' how many characters to keep. By typing :25, you tell the modifier to keep just the first 25 characters of each book title. The modifier automatically adds the ellipsis characer (...) if a word is too long and gets truncated. There is a complete list of modifiers on the SMARTY website. The Wrap Function Inserting text or numerical data into a Heurist Custom Report is easy. It is more complex to insert an image, video, audio file or location data. As a recap, consider the below code:Name: {$r.f1}
This code will create a new paragraph ( ** **), which will begin with "Name: " and then with the text from Field 1 ( **f1 **) in the relevant record ( **$r **). But what if the data you have is an image or audio file? Imagine that your custom report displays records about Persons, and you have made a recording of each Persons's voice, stored in Field 1000. You could try the following code, but it would not do the job:Voice Recording: {$r.f1000}
You might hope that this would provide a link or some other fuctionality, but instead, when users visit your website, they would see this: Voice Recording: https://heuristref.net/h6-alpha/?db=example_db&file=68e8f8ce906d1ad44eb70e97ba2b37b10cb80223 To help you with situations like this, we provide the 'wrap' function. The following code would work perfectly:Voice Recording: {wrap var=$r.f1000_originalvalue dt="file" auto_play="0"}
Voice Recording: The wrap function works with images, audio files, video files and also with simple links. If you wish users to be able to zoom in on an image or video, then you can add a 'fancybox'. To do this, add **mode **and **fancybox **parameters to the wrap: {wrap var=$r.f438_originalvalue dt="file" mode="thumbnail" fancybox="1" auto_play="0"} If $r.f438 is an image, video, pdf or similar, viewers of the custom report will now be able to click on it to zoom in and explore details. **NB: **The 'thumbnail' parameter is necessary for *videos *and *pdfs *, if you wish these to be clickable and zoomable. If you forget to write mode="thumbnail" for an image there will be no problem. You don't need to remember how to write the 'wrap' function. When you use the wizard to insert a field into your custom report, simply choose the ' **Field + function wrapper **' option before clicking ' **Insert field value **', and the 'wrap' function will be included for you automatically. For dates For custom reports the wrap function allows selection of the level of detail output for dates and what calendar to use: {wrap var=$r.f9_originalvalue dt="date" mode="1" calendar="both"} {*Date mode: 0-simple,1-full, 2-all fields; calendar: native, gregorian, both *} {$r.f9} is equivalent to {wrap var=$r.f9_originalvalue dt="date" mode="0" calendar="native"} html text fields with relative paths Images from WYSYWIG/tinymce text field are not displayed in custom reports. Relative url may be the cause ? https://dicobiosport.huma-num.fr/heurist/viewers/smarty/showReps.php?db=dicobiosport&q=id:76424&template=Record.tpl For such cases use the internal “wrap” function that converts all relative paths to absolute ones {$txt=$r.f954|regex_replace:"/\r*\n+/":""}
{wrap var=$txt}{*Biographie*}
Calculated fields Calculated fields are updated on add/save (including other records that are in list of affected record types cfn_RecTypeIDs). . Calculated fields are updated before update of record title. Calculate fields are not updated on record import. You will need to rebuild calcualted fields after import with Admin > Rebuild calculation fields Configure a ‘Weekday’ dropdown for the Mary Hamilton Project. A simple vocabulary of the seven days of the week, then defined this formula for the field: {$date = $r.f9} {$date->format(‘l’)} {date_format(date_create($r.f9),"l")} - for weekday as word {10630+date_format(date_create($r.f9),"N")} - for weekday as enum value Bootstrap Bootstrap is now incorporated as components in website format Vsn 3, but for older websites this may be useful. Is there a way of using Boostrap without messing up the CMS? CSS is not enough. Bootstrap is javascript library and affects all elements besides css. It creates its own widgets for buttons, inputs etc. Fortunately since v5 it is jquery free otherwise v4 may load its own jquery jquery-3.3.1 and it conflicts with ours OK. There is $.fn.button.noConflict(); in bootstrap that resets the appropriate element to original mode. Manual rendering of file fields You may encounter situations in which the 'wrap' function does not behave as you would wish. In such a situation, you can manually control how the file field is rendered. Click below for details. Manually accessing data in file fields: A common application of this is to include information from the description of an uploaded file in the Custom Report. For example, when uploading an image, you might include image credits in the description of the file, or a caption to be displayed, or alt text for screen readers. To include this data in your custom report, you would use the 'ulf_Description' key, like so: {$r.f38_originalvalue[0]['ulf_Description']} Linked and Related Records There are different ways that Heurist records can be linked to one another. In the simplest case, a record can have a 'record pointer' field, which simply points to another record. For example, a book may have an author field. Rather than containing a name, the 'author' field simply contains the id number of the Person who is the author of the book. Using Record Pointers in the Current Record When writing a custom report, it is easy to insert records that the current record points to, simply by using the field browser on the left of the screen. Simply choose which information you would like to include from the linked record, and use the 'insert field' tool. Heurist will insert some code that looks a bit like this: {$f1000=$heurist->getRecord($r.f1000)} Here is a detailed breakdown of the code: $f1000 ☚ The variable where you will store information about the new record. Heurist will give it a default name based on how the information is stored in the database. In this example, the book's author is stored in Field 1000, so $f1000 is used. You could change this to $author to make your code easier to read $heurist->getRecord ☚ Retrieve authors information from the database $r.f1000 ☚ The author's ID number, which is stored in Field 1000 of the book record. As the main record type in the Custom Report, the book has simply been labelled $r. If the field is repeatable, then you should click the 'repeatable' link in the field selector tool, which will insert code that looks something like this: {foreach $r.f1000s as $f1000 name=valueloop} {$f1000=$heurist->getRecord($f1000)} {* Do something with each $f1000 (i.e. author in this example *} {/foreach} This is very similar to the above code, except that there is a foreach loop, and instead of asking for the information in $r.f1000, you request information in $r.f1000s, the plural form. Linked Records The situation is more complex, however, if you wish to fetch information from *other *records that point to *this *one. For instance, suppose you want to display information about a Book. Part of the information you wish to display is information about the libraries that hold this Book. But in your database, information about library holdings is held in the Library record type. For instance, if you open up the 'New York Public Library' record, and look in the 'Books Held' field, you will see a list of all the books held by that library. When it comes time to display information about a particular book in a custom report, how can you retrieve information about all the Libraries that hold that book? To solve this problem, Heurist provides the getLinkedRecords method. The code snippet below would retrieve a list of every library that holds the current book, and then put the name of each library into a bullet-point list:Libraries holding this book:
-
{$libraries = $heurist->getLinkedRecords($r.recID, 25, 'linkedfrom')}
{foreach $libraries['linkedfrom'] as $library}
- {$library_details = $heurist->getRecord($library)} {$library_details.f1} {/foreach}
Libraries holding this book:
☚ This creates a heading for the list of libraries. You could also use a subheading element such as or
☚ This tag begins the bullet point list. Every item inside it will be included in the list. Every such item should be enclosed in - tags.
{$libraries = $heurist->getLinkedRecords($r.recID, 55, 'linkedfrom')} ☚ This line fetches information about all the libraries linked to the current book. Here is a more detailed breakdown:
$libraries ☚ The name of the variable where you will store all the libraries' ID numbers
$heurist->getLinkedRecords ☚ The method for finding linked records, which is stored inside the $heurist object
$r.recID ☚ The record ID of the current record you are looking at, which is assumed to be a book for this example
55 ☚ The RecordTypeID for the 'Library' type in this database. By putting this 55 here, you are telling Heurist only to look for *Libraries *that point to this book, as opposed to *Bookshops *or *Persons *or any other record type that may also point to Books in your database. To find the Record Type ID for a particular record, go to the Record Types tool in the Design Menu . If you don't provide a number, then Heurist will simply retrieve every record connected to this one. If you wish to search for multiple record types, you can provide an array in square brackets, e.g. [55, 66, 81].
'linkedfrom' ☚ This tells Heurist only to look for records that *point to *Books (i.e. to find records that this Book is *linked from *). You can also ask Heurist to find all records 'linkedto' this record. If you don't provide Heurist this clue, then it will simple find all records linked to this Book, whether it is the other record that points to the book, or the book that points to the other record.
{foreach $libraries['linkedfrom'] as $library} ☚ Loop over each library that this book is linked from, and do something each time
- ☚ Create a new bullet point
{$library_details = $heurist->getRecord($library)} ☚ Retrieve the current library's details
{$library_details.f1} ☚ Put the library's name in the bullet point
☚ The bullet point is now finished
{/foreach} ☚ That is all we want to do with this library – now go back to the start of the 'foreach' loop and do the same again for the next library, until all a dealt with
☚ After creating a bullet point for each library, close the list.
Related Records
A similar problem is posed by Record Relationships. These are complex interrelationships between records, and are not actually stored in the records themselves. Instead, there is a seperate table in the underlying database, which stories information about every Record Relationship. If you wish to retrieve this relationship data, we provide the $heurist->getRelatedRecords method. Let's say you are building a new custom report, displaying information about authors in your Books database. If you wished to display information about an author's relatives, you could write:
- ☚ This tag begins the bullet point list. Every item inside it will be included in the list. Every such item should be enclosed in
- tags. {$libraries = $heurist->getLinkedRecords($r.recID, 55, 'linkedfrom')} ☚ This line fetches information about all the libraries linked to the current book. Here is a more detailed breakdown: $libraries ☚ The name of the variable where you will store all the libraries' ID numbers $heurist->getLinkedRecords ☚ The method for finding linked records, which is stored inside the $heurist object $r.recID ☚ The record ID of the current record you are looking at, which is assumed to be a book for this example 55 ☚ The RecordTypeID for the 'Library' type in this database. By putting this 55 here, you are telling Heurist only to look for *Libraries *that point to this book, as opposed to *Bookshops *or *Persons *or any other record type that may also point to Books in your database. To find the Record Type ID for a particular record, go to the Record Types tool in the Design Menu . If you don't provide a number, then Heurist will simply retrieve every record connected to this one. If you wish to search for multiple record types, you can provide an array in square brackets, e.g. [55, 66, 81]. 'linkedfrom' ☚ This tells Heurist only to look for records that *point to *Books (i.e. to find records that this Book is *linked from *). You can also ask Heurist to find all records 'linkedto' this record. If you don't provide Heurist this clue, then it will simple find all records linked to this Book, whether it is the other record that points to the book, or the book that points to the other record. {foreach $libraries['linkedfrom'] as $library} ☚ Loop over each library that this book is linked from, and do something each time
- ☚ Create a new bullet point {$library_details = $heurist->getRecord($library)} ☚ Retrieve the current library's details {$library_details.f1} ☚ Put the library's name in the bullet point ☚ The bullet point is now finished {/foreach} ☚ That is all we want to do with this library – now go back to the start of the 'foreach' loop and do the same again for the next library, until all a dealt with
Author's relatives:
-
{$relatives = $heurist->getRelatedRecords($r)}
{foreach $relatives as $relative}
- {$relative.recRelationType} : {$relative.f1} {/foreach}
Bibliographical References:
-
{foreach $bibs as $bib_id=>$bib_title name=bibLoop2}
- {$bib_title} view {/foreach}