ShingleTokenFilter Class

Creates combinations of tokens as a single token. This token filter is implemented using Apache Lucene.

Constructor

ShingleTokenFilter(*args: Any, **kwargs: Any)

Variables

Name	Description
name	str The name of the token filter. It must only contain letters, digits, spaces, dashes or underscores, can only start and end with alphanumeric characters, and is limited to 128 characters. Required.
max_shingle_size	int The maximum shingle size. Default and minimum value is 2.
min_shingle_size	int The minimum shingle size. Default and minimum value is 2. Must be less than the value of maxShingleSize.
output_unigrams	bool A value indicating whether the output stream will contain the input tokens (unigrams) as well as shingles. Default is true.
output_unigrams_if_no_shingles	bool A value indicating whether to output unigrams for those times when no shingles are available. This property takes precedence when outputUnigrams is set to false. Default is false.
token_separator	str The string to use when joining adjacent tokens to form a shingle. Default is a single space (" ").
filter_token	str The string to insert for each position at which there is no token. Default is an underscore ("_").
odata_type	str A URI fragment specifying the type of token filter. Required. Default value is "#Microsoft.Azure.Search.ShingleTokenFilter".

Methods

as_dict	Return a dict that can be turned into json using json.dump.
clear	Remove all items from D.
copy
get	Get the value for key if key is in the dictionary, else default. :param str key: The key to look up. :param any default: The value to return if key is not in the dictionary. Defaults to None :returns: D[k] if k in D, else d. :rtype: any
items
keys
pop	Removes specified key and return the corresponding value. :param str key: The key to pop. :param any default: The value to return if key is not in the dictionary :returns: The value corresponding to the key. :rtype: any :raises KeyError: If key is not found and default is not given.
popitem	Removes and returns some (key, value) pair :returns: The (key, value) pair. :rtype: tuple :raises KeyError: if D is empty.
setdefault	Same as calling D.get(k, d), and setting D[k]=d if k not found :param str key: The key to look up. :param any default: The value to set if key is not in the dictionary :returns: D[k] if k in D, else d. :rtype: any
update	Updates D from mapping/iterable E and F. :param any args: Either a mapping object or an iterable of key-value pairs.
values

as_dict

Return a dict that can be turned into json using json.dump.

as_dict(*, exclude_readonly: bool = False) -> dict[str, Any]

Keyword-Only Parameters

Name	Description
exclude_readonly	bool Whether to remove the readonly properties. Default value: False

Returns

Type	Description
dict	A dict JSON compatible object

clear

Remove all items from D.

clear() -> None

copy

copy() -> Model

get

Get the value for key if key is in the dictionary, else default. :param str key: The key to look up. :param any default: The value to return if key is not in the dictionary. Defaults to None :returns: D[k] if k in D, else d. :rtype: any

get(key: str, default: Any = None) -> Any

Parameters

Name	Description
key Required
default	Default value: None

items

items() -> ItemsView[str, Any]

Returns

Type	Description
ItemsView	set-like object providing a view on D's items

keys

keys() -> KeysView[str]

Returns

Type	Description
KeysView	a set-like object providing a view on D's keys

pop

Removes specified key and return the corresponding value. :param str key: The key to pop. :param any default: The value to return if key is not in the dictionary :returns: The value corresponding to the key. :rtype: any :raises KeyError: If key is not found and default is not given.

pop(key: str, default: ~typing.Any = <object object>) -> Any

Parameters

Name	Description
key Required
default

popitem

Removes and returns some (key, value) pair :returns: The (key, value) pair. :rtype: tuple :raises KeyError: if D is empty.

popitem() -> tuple[str, Any]

setdefault

Same as calling D.get(k, d), and setting D[k]=d if k not found :param str key: The key to look up. :param any default: The value to set if key is not in the dictionary :returns: D[k] if k in D, else d. :rtype: any

setdefault(key: str, default: ~typing.Any = <object object>) -> Any

Parameters

Name	Description
key Required
default

update

Updates D from mapping/iterable E and F. :param any args: Either a mapping object or an iterable of key-value pairs.

update(*args: Any, **kwargs: Any) -> None

values

values() -> ValuesView[Any]

Returns

Type	Description
ValuesView	an object providing a view on D's values

Attributes

filter_token

The string to insert for each position at which there is no token. Default is an underscore ("_").

filter_token: str | None

max_shingle_size

The maximum shingle size. Default and minimum value is 2.

max_shingle_size: int | None

min_shingle_size

The minimum shingle size. Default and minimum value is 2. Must be less than the value of maxShingleSize.

min_shingle_size: int | None

name

The name of the token filter. It must only contain letters, digits, spaces, dashes or underscores, can only start and end with alphanumeric characters, and is limited to 128 characters. Required.

name: str

odata_type

A URI fragment specifying the type of token filter. Required. Default value is "#Microsoft.Azure.Search.ShingleTokenFilter".

odata_type: Literal['#Microsoft.Azure.Search.ShingleTokenFilter']

output_unigrams

A value indicating whether the output stream will contain the input tokens (unigrams) as well as shingles. Default is true.

output_unigrams: bool | None

output_unigrams_if_no_shingles

A value indicating whether to output unigrams for those times when no shingles are available. This property takes precedence when outputUnigrams is set to false. Default is false.

output_unigrams_if_no_shingles: bool | None

token_separator

The string to use when joining adjacent tokens to form a shingle. Default is a single space (" ").

token_separator: str | None

Feedback

Was this page helpful?