pytoda.transforms module

Transform utilities.

Summary

Classes:

AugmentByReversing	Augment an sequence by (eventually) flipping order
Compose	Composes several transforms together.
DiscardLowercase	Discard lower-case letters (and non-letter characters) from a sequence.
ExtractFromDict	Extracts a value from a dictionary based on a key set in the constructor.
LeftPadding	Left pad token indexes.
ListToTensor	2D Version of ToTensor.
Randomize	Randomize a sequence all tokens.
StartStop	Add start and stop token indexes at beginning and end of sequence.
ToTensor	Transform token indexes to torch tensor.
ToUpperCase	Convert all characters to uppercase.
Transform	Transform abstract class.

Reference

class Transform

Bases: object

Transform abstract class.

class StartStop(start_index, stop_index)

Bases: pytoda.transforms.Transform

Add start and stop token indexes at beginning and end of sequence.

__init__(start_index, stop_index)

Initialize a left padding token indexes object.

Parameters

• start_index (int) – index of start token in vocabulary.

• start_index – index of stop token in vocabulary.

class LeftPadding(padding_length, padding_index)

Bases: pytoda.transforms.Transform

Left pad token indexes.

__init__(padding_length, padding_index)

Initialize a left padding token indexes object.

Parameters

• padding_length (int) – length of the padding.

• padding_index (int) – padding index.

class ToTensor(dtype=torch.int16)

Bases: pytoda.transforms.Transform

Transform token indexes to torch tensor.

__init__(dtype=torch.int16)

Initialize a token indexes to tensor object.

Parameters

dtype (torch.dtype) – data type. Defaults to torch.short.

class ListToTensor(dtype=torch.float32)

Bases: pytoda.transforms.Transform

2D Version of ToTensor.

__init__(dtype=torch.float32)

Initialize a token indexes to tensor object.

Parameters

dtype (torch.dtype) – data type. Defaults to torch.float.

class Randomize

Bases: pytoda.transforms.Transform

Randomize a sequence all tokens.

class AugmentByReversing(p=0.5)

Bases: pytoda.transforms.Transform

Augment an sequence by (eventually) flipping order

__init__(p=0.5)

AugmentByReversing constructor.

Parameters

p (float) – Probability that reverting occurs.

class Compose(transforms)

Bases: pytoda.transforms.Transform

Composes several transforms together.

From: https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.Compose.

__init__(transforms)

Initialize a compose transform object.

Parameters

transforms (TransformList) – a list of transforms.

class ExtractFromDict(key)

Bases: pytoda.transforms.Transform

Extracts a value from a dictionary based on a key set in the constructor.

__init__(key)

Parameters

key (str) – The key used to access the dictionary

class DiscardLowercase

Bases: pytoda.transforms.Transform

Discard lower-case letters (and non-letter characters) from a sequence.

__init__()

class ToUpperCase

Bases: pytoda.transforms.Transform

Convert all characters to uppercase.