lzrw3 data compression core dual semester project april 2013 project part a final presentation...
TRANSCRIPT
LZRW3 Data Compression CoreLZRW3 Data Compression Core
Dual semester project April 2013
Project part A final presentation
Shahar Zuta Netanel Yamin
Advisor: Moshe porian
Contents Project Goals Top Architecture Micro Architecture Problems & Solutions Work method Test Plan Live simulation Synthesis Results Part B Schedule
Project GolasImplementation of LZRW3 data compression
algorithm High performance- data transfer of 1GbpsAdapted to data templates of 2Kbyte to 32KbyteInternal memory on FPGA only ( Virtex-5 ), no
interface to external memory
Implementing strong debugging capabilities
via GUI
algorithm overview
INPUT FILE
-------------------------------------------------------
A copy item consists of two bytes that represent from 3 to 18 bytes. literal item consist of one byte which represents himself
LZRW3 COMPRESSO
R
OUTPUT FILE
-]-----[]----[]-------[]-----------[]----[
GROUPS OF ITEMS(literal/Copy)
LITERALS ONLY
Top Architecture
Rx PATH
Tx PATH
INPUT BLOCK memory LZRW3
COMPRESSOR
CORE
COMPRESSED FILE memory
GUI
XILINX VIRTEX 5 ON XUVP505 BOARD
UART
UART
Top Architecture
Rx PATH
Tx PATH
INPUT BLOCKmemory LZRW3
COMPRESSOR
CORE
COMPRESSED FILE memory
GUI
XILINX VIRTEX 5 ON XUVP505 BOARD
UART
UART
Problems & solutions
Problem # 1Hash table (stage III) should not update if we
build a copy item.The decision if a copy item will be built must
consider results of the comparison(stage IV)In the original design we have four clock
cycles untill stage IV will have the comparison resultbut, we have to deal with some cases like
lzrw3ABC(index,length)ABCABC
Original design
Hash
Table
STAGE III
Address Alignmen
t
Address Alignmen
t
Tentative
Next Address
RAM BANKS
COMPARATOR
UPDATE ENABLE
STAGE IV
clk
clk
clk
clk
clk
ME
M R
EG
.
4
12
3
2
DATA
O
1
Heart of the problemDealing with cases like ABCABC mean that we
need toupdate the hash table entry with ABC offset very
fast.
ABC BCA CAB ABC
Two clocks only
:Stage III samples
2nd ABC need to read 1st
ABC offset!
1st ABC at Stage III
2nd ABC at Stage III
Solution
Hash
Table
STAGE III
Address Alignmen
t
Address Alignmen
t
Tentative
Next Address
RAM BANKS
COMPARATOR
UPDATE ENABLE
STAGE IV
clk
clk
clk
clk
clk
ME
M R
EG
.
ABC offset
1
Problem # 2The GOLDEN MODEL does not start building
a new copy item if it’s in the last 18 bytes of the file, if it’s in the middle of building a copy item it will finish it.
Solution # 1 Maintain a FIFO buffer that will hold the two
last groups, then when end of file is reached we will send the buffered bytes and not compressed bytes.
This solution would demand hardware addition and would demand thorough synchronization which could make the system unstable.
Solution # 2Taking advantage of the pipeline design:
Until the data reaches stage IV (the comparator) it goes through 9 buffers.
Buffer another 9 will complete to 18 clocks buffer before a received data will be compared.
FORWARD the EOF core input signal to stage IV and make it cause automatic comparison failure in the last 18 bytes which will result that no copy items will be created.
OUTPUT GROUP
HashFunc
Hash table
INPUT FILE
memory
COMPARATOR
4 5
HAS
H
REG
TABL
E R
EG
GRO
UP
REG
3 By
tes
buff
er
INPU
T RE
G
1234589 76
INPUT BYTE
End Of File
When EOF FARWARD UP,NEW COPY ITEMS will not
be created
Implementation
9 NEW BUFFERS
9 clocks buffer in the original design
Work methodGenerating DIRECT/RANDOM input file.Compress the input file using the GOLDEN
MODEL.Compress the same input file using our CORE.Perform a comparison between the two files using
DIFF software.
Work method (cont.)
ExamDiff
DIRECT input
RANDOM input
CCODE
GM CORE
simulation
chars
ASCII value
C
ASCII value
ExamDiff
Test Plan
Test PlanBasic set: include large amount of tests
Second set: Basic tests + Random client-ready toggles
Third set: Second set + Random inputs valid toggles
Basic set examplesRandom input (Length, Num Of Vars)
INPUTVAR.LENGTHOUTPUTLENGTH
CASE
RANDOM1032K23KReasonable compression
RANDOM1032K7407Each variable repeated 18 times
RANDOM132K3886Very high compression ratio ~90 % compression
RANDOM25632K32KOutput = input
Basic set examplesDirect input
PERFORM A COPY ITEM
COPY ITEMSHOULD
NOTPERFORM
Basic set examplesFiles concatenation
DATA PROVIDER
DATA CLIENT
BUSYFINIS
H
C L A E R
READY
LZRW3 CORE
Live simulation
Synthesis Results
Synthesis ResultsADD PLAN AHEAD/SYNPLIFY REPORT
MAX clock rateBlocks usage
Part B
Part BAssemble lzrw3 core with peripheral units Logic Test End To EndBurning to FPGA Hardware test & optimizationVerification Environment - GUI (Visual
Studio)All system Test & Debug
Schedule
Schedule
DateGoals
11/04/2013-9/05/2013
Assemble units together
12/05/2013-09/06/2013
Test logic End To End
16/06/2013 -10/06/2013
Synthesis
24/06/2013 -17/06/2013
Implementation
08/07/2013 -25/06/2013
Verification Environment – GUI
23/07/2013 -09/07/2013
Lab Tests
08/08/2013Part B Final presentation