About: Energy Complexity of Recurrent Neural Networks

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: Energy Complexity of Recurrent Neural Networks Goto Sponge NotDistinct Permalink

An Entity of Type : http://linked.opendata.cz/ontology/domain/vavai/Vysledek, within Data Space : linked.opendata.cz associated with source document(s)

Attributes	Values
rdf:type	skos:Concept http://linked.opendata.cz/ontology/domain/vavai/Vysledek
Description	Recently, a new so-called energy complexity measure has been introduced and studied for feedforward perceptron networks. This measure is inspired by the fact that biological neurons require more energy to transmit a spike than not to fire, and the activity of neurons in the brain is quite sparse, with only about 1% of neurons firing. In this paper, we investigate the energy complexity of recurrent networks which counts the number of active neurons at any time instant of a computation. We prove that any deterministic finite automaton with m states can be simulated by a neural network of optimal size s=\Theta(\sqrt{m}) with the time overhead of \tau=O(s/e) per one input bit, using the energy O(e), for any e such that e=\Omega(\log s) and e=O(s), which shows the time-energy tradeoff in recurrent networks. In addition, for the time overhead \tau satisfying \tau^\tau=o(s), we obtain the lower bound of s^{c/\tau} on the energy of such a simulation, for some constant c>0 and for infinitely many s. Recently, a new so-called energy complexity measure has been introduced and studied for feedforward perceptron networks. This measure is inspired by the fact that biological neurons require more energy to transmit a spike than not to fire, and the activity of neurons in the brain is quite sparse, with only about 1% of neurons firing. In this paper, we investigate the energy complexity of recurrent networks which counts the number of active neurons at any time instant of a computation. We prove that any deterministic finite automaton with m states can be simulated by a neural network of optimal size s=\Theta(\sqrt{m}) with the time overhead of \tau=O(s/e) per one input bit, using the energy O(e), for any e such that e=\Omega(\log s) and e=O(s), which shows the time-energy tradeoff in recurrent networks. In addition, for the time overhead \tau satisfying \tau^\tau=o(s), we obtain the lower bound of s^{c/\tau} on the energy of such a simulation, for some constant c>0 and for infinitely many s. (en)
Title	Energy Complexity of Recurrent Neural Networks Energy Complexity of Recurrent Neural Networks (en)
skos:prefLabel	Energy Complexity of Recurrent Neural Networks Energy Complexity of Recurrent Neural Networks (en)
skos:notation	RIV/67985807:_____/14:00393985!RIV15-GA0-67985807
http://linked.open...avai/riv/aktivita	I P
http://linked.open...avai/riv/aktivity	I, P(GAP202/10/1333)
http://linked.open...iv/cisloPeriodika	5
http://linked.open...vai/riv/dodaniDat	2015
http://linked.open...aciTvurceVysledku	Šíma, Jiří
http://linked.open.../riv/druhVysledku	J - Článek v odborném periodiku
http://linked.open...iv/duvernostUdaju	S - Úplné a pravdivé údaje nepodléhající ochraně podle zvláštních právních předpisů
http://linked.open...titaPredkladatele	Ústav informatiky AV ČR, v. v. i.
http://linked.open...dnocenehoVysledku	14587
http://linked.open...ai/riv/idVysledku	RIV/67985807:_____/14:00393985
http://linked.open...riv/jazykVysledku	eng - angličtina
http://linked.open.../riv/klicovaSlova	neural network; finite automaton; energy complexity; optimal size (en)
http://linked.open.../riv/klicoveSlovo	energy complexity optimal size finite automaton neural network
http://linked.open...odStatuVydavatele	US - Spojené státy americké
http://linked.open...ontrolniKodProRIV	[342FD854D9F6]
http://linked.open...i/riv/nazevZdroje	Neural Computation
http://linked.open...in/vavai/riv/obor	IN
http://linked.open...ichTvurcuVysledku	1 (xsd:int)
http://linked.open...cetTvurcuVysledku	1 (xsd:int)
http://linked.open...vavai/riv/projekt	NoSCoM: Non-Standard Computational Models and Their Applications in Complexity, Linguistics, and Learning
http://linked.open...UplatneniVysledku	2014
http://linked.open...v/svazekPeriodika	26
http://linked.open...iv/tvurceVysledku	Šíma, Jiří
http://linked.open...ain/vavai/riv/wos	000334027800005
issn	0899-7667
number of pages	21 (xsd:int)
http://bibframe.org/vocab/doi	10.1162/NECO_a_00579

Faceted Search & Find service v1.16.118 as of Jun 21 2024

Alternative Linked Data Documents: ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3240 as of Jun 21 2024, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (126 GB total memory, 97 GB memory in use)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2024 OpenLink Software