Alex Coyte
952f0aad21
support dynamic stream compression
2017-08-21 11:18:54 +10:00
Wang, Xiang W
252eb820c4
ue-3145: make parents of included literals exclusive
2017-08-21 11:12:36 +10:00
Wang, Xiang W
86c5f7feb1
FDR: Squash buckets of included literals in FDR confirm
...
- Change the compile of literal matchers to two passes.
- Reverse the bucket assignment in FDR, bucket with longer literals has
smaller bucket id.
- Squash the buckets of included literals and jump to the the program of
included literals directly from parent literal program without going
through FDR confirm for included iterals.
2017-08-21 11:12:36 +10:00
Justin Viiret
bf3ced92f4
hwlm_dump: take base filename, like NFA dump API
2017-08-21 11:10:11 +10:00
Justin Viiret
f762fb9af6
dump_util: richer StdioFile type
2017-08-21 11:10:11 +10:00
Justin Viiret
4edf1e4195
dump: move openStdioFile() to util/dump_util.h
2017-08-21 11:10:11 +10:00
Justin Viiret
bc953717c1
rose: dump lit tables in their own files
2017-08-21 11:10:11 +10:00
Justin Viiret
bdae3d5b80
dump: always allocate >=8 bytes for multibit
...
The multibit runtime assumes that it is always safe to read 8 bytes, so
we must over-allocate for smaller sizes. Caught by ASan.
2017-08-21 10:40:18 +10:00
Alex Coyte
15c8a7bd98
rose: rework storage of extra lookaround information
...
- remove explicit lookaround table from bytecode
- make the RoseInstr responsible for adding required info to blob
2017-05-30 13:59:00 +10:00
Justin Viiret
8b9328fe9e
rose: replace RoseLiteralMap use of bimap
...
This apoproach is simpler and more efficient for cases with large
numbers of literals.
2017-05-30 13:58:59 +10:00
Alex Coyte
f74f475189
rose_program: merge RECORD_ANCHORED instruction into ANCHORED_DELAY
2017-05-30 13:58:32 +10:00
Alex Coyte
88fd95e38a
rose: minor clean up of catchup
...
- anchored dfa do not mean that catchup is required
- remove needsCatchup from rose bytecode as catchup is based on interpreter
2017-05-30 13:58:32 +10:00
Alex Coyte
37cb93e60f
rose_build: reduce size/scope of context objects
2017-04-26 15:19:01 +10:00
Xu, Chi
ae3cb7de6f
rose: add multi-path shufti 16x8, 32x8, 32x16, 64x8 and multi-path lookaround instructions.
2017-04-26 15:18:56 +10:00
Justin Viiret
18f843bcc1
rose: add CLEAR_WORK_DONE instruction
...
Preparatory work for allowing fragments to be shared between literals
that squash groups and those that don't.
2017-04-26 15:18:26 +10:00
Alex Coyte
de52b30c3e
make rose responsible for dumping its bytecode
2017-04-26 15:17:19 +10:00
Justin Viiret
e3d2d67833
rose: move lookaround tables to engine blob
2017-04-26 15:11:10 +10:00
Justin Viiret
bf93c993cb
rose: remove final_id
2017-04-26 15:04:31 +10:00
Justin Viiret
a06e877fde
rose dump: simplify
2017-04-26 15:04:31 +10:00
Justin Viiret
bcbd85ab67
rose: dump support for delay programs
2017-04-26 15:04:31 +10:00
Justin Viiret
6a945e27fb
rose: reduce delay program dep on final_id
2017-04-26 15:04:31 +10:00
Justin Viiret
dc8220648c
rose: remove now-unused anchored_base_id
2017-04-26 15:04:30 +10:00
Justin Viiret
79512bd5c3
rose: use fragment ids earlier for anchored dfas
2017-04-26 15:04:30 +10:00
Justin Viiret
821a1b81e6
rose: only dump delay rebuild table if streaming
2017-04-26 14:58:46 +10:00
Justin Viiret
a5b3bc814f
rose: delete RoseEngine::literalCount
2017-04-26 14:49:51 +10:00
Justin Viiret
9550058e75
remove lit program tables from bytecode
2017-04-26 14:49:51 +10:00
Justin Viiret
bd3357d3ac
rose: dump lit programs from frag map
2017-04-26 14:49:51 +10:00
Justin Viiret
c6bf1919d0
rose: merge all dump code into rose_build_dump.cpp
2017-04-26 14:49:35 +10:00
Justin Viiret
c2cac5009a
tidy up args to builders
2017-04-26 14:46:49 +10:00
Justin Viiret
3ae2fb417e
move final_to_frag_map into RoseBuildImpl (for dump code)
2017-04-26 14:46:49 +10:00
Justin Viiret
ac858cd47c
rose: build a separate delay rebuild matcher
2017-04-26 14:46:48 +10:00
Justin Viiret
eb14792a63
rose: group final ids by fragment
2017-04-26 14:41:29 +10:00
Justin Viiret
07a6b6510c
rose/hwlm: limit literals to eight bytes
...
Rework HWLM to work over literals of eight bytes ("medium length"),
doing confirm in the Rose interpreter.
2017-04-26 14:41:29 +10:00
Justin Viiret
68bdc800fc
dump: render literals as regexes (with comments)
2017-04-26 14:41:29 +10:00
Alex Coyte
e1e9010cac
Introduce custom adjacency-list based graph
2016-12-02 11:31:33 +11:00
Justin Viiret
68bf473e2e
fdr: move long literal handling into Rose
...
Move the hash table used for long literal support in streaming mode from
FDR to Rose, and introduce new instructions CHECK_LONG_LIT and
CHECK_LONG_LIT_NOCASE for doing literal confirm for long literals.
This simplifies FDR confirm, and guarantees that HWLM matchers will only
be used for literals < 256 bytes long.
2016-10-28 14:52:26 +11:00
Alex Coyte
5c5ec905cc
violet: initial implementation
2016-08-10 15:01:08 +10:00
Alex Coyte
b13a90e5d2
compiledump: allow disabling of early graphs for large compiles
2016-08-10 15:00:44 +10:00
Justin Viiret
6ec93a54c4
rose: only dump small-block table in block mode
2016-08-10 14:57:20 +10:00
Justin Viiret
39c6a0c7bf
rose: check literal bounds when building SB table
...
Literals that cannot lead to a report in the first ROSE_SMALL_BLOCK_LEN
bytes may be dropped from the small block table.
2016-08-10 14:56:12 +10:00
Alex Coyte
f166bc5658
allow some prefixes that may squash the literal match to run eagerly
2016-07-08 11:01:34 +10:00
Justin Viiret
9b7eca5400
rose: dump leftfix/suffix queue indices
2016-07-08 10:44:56 +10:00
Justin Viiret
1f41a921f2
mcclellan, gough: native report remapping
2016-05-18 16:20:45 +10:00
Justin Viiret
b093616aff
Rose build: move HWLM build code to own file
...
To reduce the size of rose_build_bytecode.cpp a little, move the code
that deals with HWLM literal tables into its own new file.
2016-04-20 13:34:54 +10:00
Justin Viiret
67b9784dae
Rose: use program for all literal matches
...
Unifies all literal match paths so that the Rose program is used for all
of them. This removes the previous specialised "direct report" and
"multi direct report" paths. Some additional REPORT instruction work was
necessary for this.
Reworked literal construction path at compile time in prep for using
program offsets as literal IDs.
Completely removed the anchored log runtime, which is no longer worth
the extra complexity.
2016-04-20 13:34:54 +10:00
Justin Viiret
48c9d7c381
Remove use of depth from Rose entirely
2016-03-01 11:23:11 +11:00
Justin Viiret
b2ebdac642
rose: Extend program to handle literals, iterators
...
- cleanups
- add sparse iter instructions
- merge "root" and "sparse iter" programs together
- move program execution to new file program_runtime.h
- simplify EOD execution
2016-03-01 11:17:31 +11:00
Justin Viiret
d67c7583ea
rose: Extend the interpreter to handle more work
...
- Use program for EOD sparse iterator
- Use program for literal sparse iterator
- Eliminate RoseRole, RosePred, RoseVertexProps::role
- Small performance optimizations
2016-03-01 11:16:02 +11:00
Justin Viiret
9cb2233589
rose: Use an interpreter for role runtime
...
Replace much of the RoseRole structure with an interpreted program,
simplifying the Rose runtime and making it much more flexible.
2016-03-01 11:16:02 +11:00
Alex Coyte
a7d8dafb71
detach the sidecar
2016-03-01 11:13:23 +11:00
