From 8405e505c4ff057af0bfdeae1c3bf6cc210f005d Mon Sep 17 00:00:00 2001 From: Teodor Sigaev Date: Mon, 4 Aug 2003 14:11:08 +0000 Subject: [PATCH] Docs fixes --- contrib/tsearch2/docs/tsearch2-ref.html | 264 ++++++++++++------------ 1 file changed, 137 insertions(+), 127 deletions(-) diff --git a/contrib/tsearch2/docs/tsearch2-ref.html b/contrib/tsearch2/docs/tsearch2-ref.html index df0faa47d9..acb673bcab 100644 --- a/contrib/tsearch2/docs/tsearch2-ref.html +++ b/contrib/tsearch2/docs/tsearch2-ref.html @@ -1,67 +1,63 @@ - - - - -tsearch2 reference - + +tsearch2 reference + -

The tsearch2 Reference

+

The tsearch2 Reference

-

-Brandon Craig Rhodes
30 June 2003 -

+

+Brandon Craig Rhodes
30 June 2003 (edited by Oleg Bartunov, 2 Aug 2003). +

This Reference documents the user types and functions of the tsearch2 module for PostgreSQL. An introduction to the module is provided -by the tsearch2 Guide, +by the tsearch2 Guide, a companion document to this one. You can retrieve a beta copy of the tsearch2 module from the -GiST for PostgreSQL -page — look under the section entitled Development History +GiST for PostgreSQL +page -- look under the section entitled Development History for the current version. -

Vectors and Queries

+

Vectors and Queries

-Vectors and queries both store lexemes, +Vectors and queries both store lexemes, but for different purposes. A tsvector stores the lexemes of the words that are parsed out of a document, and can also remember the position of each word. A tsquery specifies a boolean condition among lexemes. -

-Any of the following functions with a configuration argument +

+Any of the following functions with a configuration argument can use either an integer id or textual ts_name to select a configuration; if the option is omitted, then the current configuration is used. For more information on the current configuration, read the next section on Configurations. -

Vector Operations

+

Vector Operations

-
-
- to_tsvector( [configuration,] +
+ to_tsvector( [configuration,] document TEXT) RETURNS tsvector -
- Parses a document into tokens, +
+ Parses a document into tokens, reduces the tokens to lexemes, and returns a tsvector which lists the lexemes together with their positions in the document. For the best description of this process, - see the section on Parsing and Stemming + see the section on Parsing and Stemming in the accompanying tsearch2 Guide. -
+
strip(vector tsvector) RETURNS tsvector -
+
Return a vector which lists the same lexemes as the given vector, but which lacks any information about where in the document each lexeme appeared. While the returned vector is thus useless for relevance ranking, it will usually be much smaller. -
+
setweight(vector tsvector, letter) RETURNS tsvector -
+
This function returns a copy of the input vector in which every location has been labelled with either the letter @@ -72,12 +68,12 @@ read the next section on Configurations. These labels are retained when vectors are concatenated, allowing words from different parts of a document to be weighted differently by ranking functions. -
+
vector1 || vector2 -
+
concat(vector1 tsvector, vector2 tsvector) RETURNS tsvector -
+
Returns a vector which combines the lexemes and position information in the two vectors given as arguments. Position weight labels (described in the previous paragraph) @@ -98,53 +94,52 @@ read the next section on Configurations. and then providing a weights argument to the rank() function that assigns different weights to positions with different labels. -
+
tsvector_size(vector tsvector) RETURNS INT4 -
+
Returns the number of lexemes stored in the vector. -
+
text::tsvector RETURNS tsvector -
+
Directly casting text to a tsvector allows you to directly inject lexemes into a vector, with whatever positions and position weights you choose to specify. The text should be formatted like the vector would be printed by the output of a SELECT. - See the Casting + See the Casting section in the Guide for details. -
+

Query Operations

-
-
+
to_tsquery( [configuration,] querytext text) RETURNS tsvector -
+
Parses a query, which should be single words separated by the boolean operators - “&” and, - “|” or, - and “!” not, + "&" and, + "|" or, + and "!" not, which can be grouped using parenthesis. Each word is reduced to a lexeme using the current or specified configuration. - -
+ +
querytree(query tsquery) RETURNS text -
+
This might return a textual representation of the given query. -
+
text::tsquery RETURNS tsquery -
+
Directly casting text to a tsquery allows you to directly inject lexemes into a query, with whatever positions and position weight flags you choose to specify. The text should be formatted like the query would be printed by the output of a SELECT. - See the Casting + See the Casting section in the Guide for details. -
+

Configurations

@@ -157,39 +152,38 @@ uses a configuration to perform its processing. Three configurations come with tsearch2: + The tsearch2 modules initially chooses your current configuration by looking for your current locale in the locale field of the pg_ts_cfg table described below. You can manipulate the current configuration yourself with these functions: -
-
+
set_curcfg( id INT | ts_name TEXT ) RETURNS VOID -
+
Set the current configuration used by to_tsvector and to_tsquery. -
+
show_curcfg() RETURNS INT4 -
+
Returns the integer id of the current configuration. -
+

Each configuration is defined by a record in the pg_ts_cfg table: -

create table pg_ts_cfg (
+
create table pg_ts_cfg (
 	id		int not  null primary key,
 	ts_name		text not null,
 	prs_name	text not null,
@@ -200,17 +194,17 @@ The id and ts_name are unique values
 which identify the configuration;
 the prs_name specifies which parser the configuration uses.
 Once this parser has split document text into tokens,
-the type of each resulting token —
-or, more specifically, the type's lex_alias
-as specified in the parser's lexem_type() table —
+the type of each resulting token --
+or, more specifically, the type's tok_alias
+as specified in the parser's lexem_type() table --
 is searched for together with the configuration's ts_name
 in the pg_ts_cfgmap table:
 
 
create table pg_ts_cfgmap (
 	ts_name		text not null,
-	lex_alias	text not null,
+	tok_alias	text not null,
 	dict_name	text[],
-	primary key (ts_name,lex_alias)
+	primary key (ts_name,tok_alias)
 );

 
 Those tokens whose types are not listed are discarded.
@@ -227,17 +221,16 @@ or discarding the token if no dictionary returns a lexeme for it.
 Each parser is defined by a record in the pg_ts_parser table:
 
 
create table pg_ts_parser (
-	prs_id		int not null primary key,
 	prs_name	text not null,
 	prs_start	oid not null,
-	prs_getlexem	oid not null,
+	prs_nexttoken	oid not null,
 	prs_end		oid not null,
 	prs_headline	oid not null,
 	prs_lextype	oid not null,
 	prs_comment	text
 );

 
-The prs_id and prs_name uniquely identify the parser,
+The prs_name uniquely identify the parser,
 while prs_comment usually describes its name and version
 for the reference of users.
 The other items identify the low-level functions
@@ -246,40 +239,65 @@ and are only of interest to someone writing a parser of their own.
 

 The tsearch2 module comes with one parser named default
 which is suitable for parsing most plain text and HTML documents.
-

+

 Each parser argument below
-must designate a parser with either an integer prs_id
-or a textual prs_name;
+must designate a parser with prs_name;
 the current parser is used when this argument is omitted.
 
-

-

+

  CREATE FUNCTION set_curprs(parser) RETURNS VOID
-

+

  Selects a current parser
  which will be used when any of the following functions
  are called without a parser as an argument.
-

- CREATE FUNCTION lexem_type(
+

+ CREATE FUNCTION token_type(
   [ parser ]
-  ) RETURNS SETOF lexemtype
-

+  ) RETURNS SETOF tokentype
+

  Returns a table which defines and describes
  each kind of token the parser may produce as output.
- For each token type the table gives the lexid
+ For each token type the table gives the tokid
  which the parser will label each token of that type,
  the alias which names the token type,
  and a short description descr for the user to read.
-

+ 

+ Example:
+ 

+ 
 apod=# select m.ts_name, t.alias as tok_type, t.descr as description, p.token,\
+ apod=# m.dict_name, strip(to_tsvector(p.token)) as tsvector\
+ apod=# from parse('Tsearch module for PostgreSQL 7.3.3') as\
+ apod=# p, token_type() as t, pg_ts_cfgmap as m, pg_ts_cfg as c\
+ apod=# where t.tokid=p.tokid and t.alias = m.tok_alias\
+ apod=# and m.ts_name=c.ts_name and c.oid=show_curcfg();
+  ts_name | tok_type | description |   token    | dict_name |  tsvector    
+ ---------+----------+-------------+------------+-----------+--------------
+  default | lword    | Latin word  | Tsearch    | {en_stem} | 'tsearch'
+  default | word     | Word        | module     | {simple}  | 'modul'
+  default | lword    | Latin word  | for        | {en_stem} | 
+  default | lword    | Latin word  | PostgreSQL | {en_stem} | 'postgresql'
+  default | version  | VERSION     | 7.3.3      | {simple}  | '7.3.3'
+ 

+ Here:
+ 
    
+ 
  •  tsname - configuration name
+ 
  •  tok_type  - token type
+ 
  •  description - human readable name of tok_type
+ 
  •  token       - parser's token
+ 
  •  dict_name - dictionary will be used for the token
+ 
  •  tsvector - final result
+ 

+
+

  CREATE FUNCTION parse(
   [ parser, ] document TEXT
-  ) RETURNS SETOF lexemtype
-

+  ) RETURNS SETOF tokenout
+

  Parses the given document and returns a series of records,
  one for each token produced by parsing.
- Each token includes a lexid giving its type
+ Each token includes a tokid giving its type
  and a lexem which gives its content.
-

+
 
 
Dictionaries

 
@@ -291,24 +309,23 @@ Among the dictionaries which come installed with tsearch2 are:
 
    
 
  • simple simply folds uppercase letters to lowercase
  before returning the word.
-
  • en_stem runs an English Snowball stemmer on each word
+
  • en_stem runs an English Snowball stemmer on each word
  that attempts to reduce the various forms of a verb or noun
  to a single recognizable form.
-
  • ru_stem runs a Russian Snowball stemmer on each word.
-

+
  • ru_stem runs a Russian Snowball stemmer on each word.
+
    • 
 
 Each dictionary is defined by an entry in the pg_ts_dict table:
 
 
    CREATE TABLE pg_ts_dict (
-	dict_id		int not null primary key,
 	dict_name	text not null,
 	dict_init	oid,
 	dict_initoption	text,
-	dict_lemmatize	oid not null,
+	dict_lexize	oid not null,
 	dict_comment	text
 );
    
 
-The dict_id and dict_name
+The dict_name
 serve as unique identifiers for the dictionary.
 The meaning of the dict_initoption varies among dictionaries,
 but for the built-in Snowball dictionaries
@@ -319,33 +336,32 @@ useful only to developers trying to implement their own dictionaries.
 
    
 The argument named dictionary
 in each of the following functions
-should be either an integer dict_id or a textual dict_name
+should be dict_name
 identifying which dictionary should be used for the operation;
 if omitted then the current dictionary is used.
 
-
    
-
    
+
    
  CREATE FUNCTION set_curdict(dictionary) RETURNS VOID
-
    
+
    
  Selects a current dictionary for use by functions
  that do not select a dictionary explicitly.
-
    
+
    
  CREATE FUNCTION lexize(
  [ dictionary, ] word text)
  RETURNS TEXT[]
-
    
+
    
  Reduces a single word to a lexeme.
  Note that lexemes are arrays of zero or more strings,
  since in some languages there might be several base words
  from which an inflected form could arise.
-
    
+
 
 
    Ranking
    
 
 Ranking attempts to measure how relevant documents are to particular queries
 by inspecting the number of times each search word appears in the document,
 and whether different search terms occur near each other.
-Note that this information is only available in unstripped vectors —
+Note that this information is only available in unstripped vectors --
 ranking functions will only return a useful result
 for a tsvector which still has position information!
 
    
@@ -357,45 +373,42 @@ since a hundred-word document with five instances of a search word
 is probably more relevant than a thousand-word document with five instances.
 The option can have the values:
 
-
      
+
        
 
      • 0 (the default) ignores document length.
-
      • 1 divides the rank by the logarithm of the length.
-
      • 2 divides the rank by the length itself.
-
      
+
    • 1 divides the rank by the logarithm of the length.
+
    • 2 divides the rank by the length itself.
+
    
 
 The two ranking functions currently available are:
 
-
    
-
    
+
    
  CREATE FUNCTION rank(
    
   [ weights float4[], ]
   vector tsvector, query tsquery,
   [ normalization int4 ]
    
   ) RETURNS float4    
-
    
+
    
  This is the ranking function from the old version of OpenFTS,
  and offers the ability to weight word instances more heavily
  depending on how you have classified them.
  The weights specify how heavily to weight each category of word:
- 
    {D-weight, A-weight, B-weight, C-weight}
    
+ 
    {D-weight, C-weight, B-weight, A-weight}
    
  If no weights are provided, then these defaults are used:
  
    {0.1, 0.2, 0.4, 1.0}
    
  Often weights are used to mark words from special areas of the document,
  like the title or an initial abstract,
  and make them more or less important than words in the document body.
-
    
+
    
  CREATE FUNCTION rank_cd(
    
   [ K int4, ]
   vector tsvector, query tsquery,
   [ normalization int4 ]
    
   ) RETURNS float4    
-
    
+
    
  This function computes the cover density ranking
  for the given document vector and query,
  as described in Clarke, Cormack, and Tudhope's
- “Relevance Ranking for One to Three Term Queries”
+ "Relevance Ranking for One to Three Term Queries"
  in the 1999 Information Processing and Management.
  The value K is one of the values from their formula,
  and defaults to K=4.
@@ -403,18 +416,17 @@ The two ranking functions currently available are:
  we can roughly describe the term
  as stating how far apart two search terms can fall
  before the formula begins penalizing them for lack of proximity.
-
    
+
 
 
    Headlines
    
 
-
    
-
    
+
    
  CREATE FUNCTION headline(
    
   [ id int4, | ts_name text, ]
   document text, query tsquery,
   [ options text ]
    
   ) RETURNS text    
-
    
+
    
  Every form of the the headline() function
  accepts a document along with a query,
  and returns one or more ellipse-separated excerpts from the document
@@ -424,25 +436,23 @@ The two ranking functions currently available are:
  if none is specified that the current configuration is used instead.
  
    
  An options string if provided should be a comma-separated list
- of one or more ‘option=value’ pairs.
+ of one or more 'option=value' pairs.
  The available options are:
- 
      
-  
    • StartSel, StopSel —
+ 
        
+  
      • StartSel, StopSel --
    the strings with which query words appearing in the document
    should be delimited to distinguish them from other excerpted words.
-  
      • MaxWords, MinWords —
+  
      • MaxWords, MinWords --
    limits on the shortest and longest headlines you will accept.
-  
      • ShortWord —
+  
      • ShortWord --
    this prevents your headline from beginning or ending
    with a word which has this many characters or less.
    The default value of 3 should eliminate most English
    conjunctions and articles.
- 
      
+ 
    
  Any unspecified options receive these defaults:
- 
    -StartSel=<b>, StopSel=</b>, MaxWords=35, MinWords=15, ShortWord=3
+ 
    StartSel=<b>, StopSel=</b>, MaxWords=35, MinWords=15, ShortWord=3
  
    
-
    
+
 
-
-
+
\ No newline at end of file
-- 
2.40.0