segmenter

Intl Explorer is an interactive tool for experimenting and trying out the ECMAScript Internationalization API.

Transport stream web based HLS segmenter.

Linux HLS Server including uploader, segmenter, chunks dealer and media manager

Tokenization, sentence segmentation, POS tagging and dependency parsing for biomedical texts (BMC Bioinformatics 2019)

A high-performance wrapper around Intl.Segmenter for efficient text segmentation. This class resolves memory handling issues seen with large strings and "maximum call stack exceeded" exceptions that occur when strings exceed 40-50k characters. Enhances performance by 50-500x. Only ~70 loc (with comments) and no dependencies.

Chinese Tokenizer module for Python

Text segmenter and tokeniser for Danish, English and other languages. Reads an RTF or flat text file and outputs the text, one line per sentence & optionally tokenized.

Segmenter and Orthography Standardazier (SOS) for Classical Arabic (CA)

This is a tool that allows you to create an DASH manifest from any fmp4 stream file. There are online and CLI versions, also we provide file and stream versions

[VISAPP 2026] Graph-Guided Token Merging (G2TM) is a lightweight one-shot module designed to eliminate redundant tokens in the early layers of a ViT-based models, through graph theory. It performs a single token merging step after a shallow attention block, enabling all subsequent layers to operate on a compact token set.

Data for layout analysis and HTR.

Dataset and models for catalogs' Layout analysis and HTR

Tool for visual timestamp annotation, automated season fingerprinting, and uploading segments to TheIntroDB and IntroDB.

An extension of Intl.Segmenter that supports ANSI escape codes – with zero dependencies.

A rule-based sentence segmenter (splitter) and a word tokenizer using orthographic features

jieba中文分词库jieba.NET的.NET Standard版本

Text tokenizer for linguistic purposes, such as text matching. Supports more than 40 languages, including English, French, Russian, Japanese, Thai etc.

A lightweight, intuitive wrapper around Intl.Segmenter for seamless segment-aware string operations in TypeScript and JavaScript

A native macOS SwiftUI app to split long text into clean, copy-ready segments — with smart logic and multilingual support.

Sequence tagging(Word Segmenter), POS tagging and NER with Tensorflow