About: Fast Construction of a Word-Number Index for Large Data

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: Fast Construction of a Word-Number Index for Large Data Goto Sponge NotDistinct Permalink

An Entity of Type : http://linked.opendata.cz/ontology/domain/vavai/Vysledek, within Data Space : linked.opendata.cz associated with source document(s)

Attributes	Values
rdf:type	skos:Concept http://linked.opendata.cz/ontology/domain/vavai/Vysledek
Description	The paper presents a work still in progress, but with promising results. We offer a new method of construction of word to number and number to word indices for very large corpus data (tens of billions of tokens), which is up to an order of magnitude faster than the current approach. We use HAT-trie for sorting the data and Daciuk’s algorithm for building a minimal deterministic finite state automaton from sorted data. The latter we reimplemented and our new implementation is roughly three times faster and with smaller memory footprint than the one of Daciuk. This is useful not only for building word-number indices, but also for many other applications, e.g. building data for morphological analysers. The paper presents a work still in progress, but with promising results. We offer a new method of construction of word to number and number to word indices for very large corpus data (tens of billions of tokens), which is up to an order of magnitude faster than the current approach. We use HAT-trie for sorting the data and Daciuk’s algorithm for building a minimal deterministic finite state automaton from sorted data. The latter we reimplemented and our new implementation is roughly three times faster and with smaller memory footprint than the one of Daciuk. This is useful not only for building word-number indices, but also for many other applications, e.g. building data for morphological analysers. (en)
Title	Fast Construction of a Word-Number Index for Large Data Fast Construction of a Word-Number Index for Large Data (en)
skos:prefLabel	Fast Construction of a Word-Number Index for Large Data Fast Construction of a Word-Number Index for Large Data (en)
skos:notation	RIV/00216224:14330/13:00070327!RIV14-MSM-14330___
http://linked.open...avai/riv/aktivita	P S
http://linked.open...avai/riv/aktivity	P(LM2010013), S
http://linked.open...vai/riv/dodaniDat	2014
http://linked.open...aciTvurceVysledku	Šmerk, Pavel Jakubíček, Miloš Rychlý, Pavel
http://linked.open.../riv/druhVysledku	D - Článek ve sborníku
http://linked.open...iv/duvernostUdaju	S - Úplné a pravdivé údaje nepodléhající ochraně podle zvláštních právních předpisů
http://linked.open...titaPredkladatele	Masarykova univerzita / Fakulta informatiky
http://linked.open...dnocenehoVysledku	74703
http://linked.open...ai/riv/idVysledku	RIV/00216224:14330/13:00070327
http://linked.open...riv/jazykVysledku	eng - angličtina
http://linked.open.../riv/klicovaSlova	word to number index; number to word index; finite state automata; hat-trie (en)
http://linked.open.../riv/klicoveSlovo	finite state automata hat-trie number to word index word to number index
http://linked.open...ontrolniKodProRIV	[7EBD1E1A47FE]
http://linked.open...v/mistoKonaniAkce	Brno
http://linked.open...i/riv/mistoVydani	Brno
http://linked.open...i/riv/nazevZdroje	RASLAN 2013 Recent Advances in Slavonic Natural Language Processing
http://linked.open...in/vavai/riv/obor	IN
http://linked.open...ichTvurcuVysledku	3 (xsd:int)
http://linked.open...cetTvurcuVysledku	3 (xsd:int)
http://linked.open...vavai/riv/projekt	LINDAT-CLARIN: Institute for analysis, processing and distribution of linguistic data
http://linked.open...UplatneniVysledku	2013
http://linked.open...iv/tvurceVysledku	Jakubíček, Miloš Šmerk, Pavel Rychlý, Pavel
http://linked.open...vavai/riv/typAkce	CST - Celostátní
http://linked.open.../riv/zahajeniAkce	2013-01-01 (xsd:date)
number of pages	5 (xsd:int)
http://purl.org/ne...btex#hasPublisher	Tribun EU
https://schema.org/isbn	9788026305200
http://localhost/t...ganizacniJednotka	14330

Faceted Search & Find service v1.16.118 as of Jun 21 2024

Alternative Linked Data Documents: ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3240 as of Jun 21 2024, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (126 GB total memory, 48 GB memory in use)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2024 OpenLink Software