Compare commits

...

9 Commits

16 changed files with 501 additions and 75 deletions

3
.gitignore vendored
View File

@ -1,2 +1,3 @@
__pycache__
*.jpg
*.jpg
*.png

181
README.md Normal file
View File

@ -0,0 +1,181 @@
<h3>rivet - Register / Instruction Visualizer and Explainer Tool</h3>
---
## Table of contents
<!-- TOC -->
* [Table of contents](#table-of-contents)
* [Introduction](#introduction)
* [Requirements](#requirements)
* [Usage](#usage)
* [Options](#options)
* [Examples](#examples)
* [config.json](#configjson)
* [dark.json](#darkjson)
* [blueprint.json](#blueprintjson)
* [transparent.json](#transparentjson)
* [Config](#config)
<!-- TOC -->
---
## Introduction
This tool lets you generate visual explanations of binary instructions, or describe the contents of a register.
The layout and style can be customized to fit your needs.
For a full description of the schema format, please check out [format.md](format.md)
## Requirements
- [Python 3+](https://www.python.org/)
- [Pygame](https://pypi.org/project/pygame/) - used to render the images
- [Beautiful Soup 4](https://pypi.org/project/beautifulsoup4/) - used to parse XML files
- [PyYAML](https://pypi.org/project/PyYAML/) - used to parse YAML files
## Usage
Basic usage:
```bash
python3 main.py schema.xml
```
Directory mode + config:
```bash
python3 main.py -o out/ -c config.json -d schemas/
```
## Options
Several command line options are available:
- `-o PATH` sets the output path. If not given, the output file will be located in the same directory as the input file, and given the same name (different extension)
- `-d` enables directory mode. If set, the input and output paths are directories and all files inside the input directory are processed
- `-c PATH` sets the config path. For more details on available config parameters, please read the [Config section](#config).
## Examples
The following images were generated from this schema ([example1.yaml](example1.yaml)):
<details>
<summary>Show / hide</summary>
```yaml
structures:
main:
bits: 32
ranges:
31-28:
name: cond
27:
name: 0
26:
name: 1
25:
name: I
24:
name: P
description: pre / post indexing bit
values:
0: post, add offset after transfer
1: pre, add offset before transfer
23:
name: U
description: up / down bit
values:
0: down, subtract offset from base
1: up, addition offset to base
22:
name: B
description: byte / word bit
values:
0: transfer word quantity
1: transfer byte quantity
21:
name: W
description: write-back bit
values:
0: no write-back
1: write address into base
20:
name: L
description: load / store bit
values:
0: store to memory
1: load from memory
19-16:
name: Rn
description: base register
15-12:
name: Rd
description: source / destination register
11-0:
name: offset
depends-on: 25
values:
0:
description: offset is an immediate value
structure: immediateOffset
1:
description: offset is a register
structure: registerOffset
immediateOffset:
bits: 12
ranges:
11-0:
name: 12-bit immediate offset
description: unsigned number
registerOffset:
bits: 12
ranges:
11-4:
name: shift
description: shift applied to Rm
3-0:
name: Rm
description: offset register
```
</details>
#### config.json
![default black on white](example_normal.png)
#### dark.json
![white on black](example_dark.png)
#### blueprint.json
![white on blue](example_blueprint.png)
#### transparent.json
![grey on transparent](example_transparent.png)
## Config
The config file may change the following values to customize the layout and style:
- `defaultFontFamily`: the default font family
- `defaultFontSize`: the default font size
- `italicFontFamily`: the italic font family (for value description)
- `italicFontSize`: the italic font size
- `backgroundColor`: the image background color (ex: [222, 250, 206])
- `textColor`: the default text color
- `linkColor`: the color of linking lines and arrows
- `borderColor`: the color of register borders
- `bitWidth`: the width of 1 bit (in pixels)
- `bitHeight`: the height of 1 bit (in pixels)
- `descriptionMargin`: the space between descriptions (in pixels)
- `dashLength`: the length of one dash (for dashed lines)
- `dashSpace`: the space between dashes (for dashed lines)
- `arrowSize`: the arrow size (height in pixels, width for horizontal arrow)
- `margins`: the margins from the borders of the image (in pixels, [top, right, bottom, left])
- `arrowMargin`: the margin between arrows and registers (in pixels)
- `valuesGap`: the gap between values (in pixels)
- `arrowLabelDistance`: the distance between arrows and their label (in pixels)
- `forceDescsOnSide`: if true, descriptions are placed on the side of the register, otherwise, they are placed as close as possible to the bit
- `leftLabels`: if true, descriptions are put on the left, otherwise, they default to the right hand side
- `width`: the image width (in pixels)
- `height`: the image height (in pixels)
Some config examples are already included:
- [config.json](config.json): Default configuration values (black on white)
- [dark.json](dark.json): Dark theme (white on black)
- [blueprint.json](blueprint.json): Blueprint theme (white on blue)
- [transparent.json](transparent.json): Transparent background, for easy integration in other documents (grey on transparent)

View File

@ -17,5 +17,9 @@
"margins": [20, 20, 20, 20],
"arrowMargin": 4,
"valuesGap": 5,
"arrowLabelDistance": 5
"arrowLabelDistance": 5,
"forceDescsOnSide": false,
"leftLabels": false,
"width": 1200,
"height": 800
}

View File

@ -1,6 +1,9 @@
from __future__ import annotations
import json
import re
class Config:
DEFAULT_FONT_FAMILY = "Ubuntu Mono"
DEFAULT_FONT_SIZE = 16
@ -21,11 +24,16 @@ class Config:
ARROW_MARGIN = 4
VALUES_GAP = 5
ARROW_LABEL_DISTANCE = 5
FORCE_DESCS_ON_SIDE = False
LEFT_LABELS = False
WIDTH = 1200
HEIGHT = 800
def __init__(self, path: str = "config.json") -> None:
self.load(path)
def load(self, path: str) -> None:
@staticmethod
def load(path: str) -> None:
with open(path, "r") as f:
config = json.load(f)
@ -34,5 +42,6 @@ class Config:
if hasattr(Config, k):
setattr(Config, k, v)
@staticmethod
def formatKey(key: str) -> str:
return re.sub(r"([a-z])([A-Z])", r"\1_\2", key).upper()
return re.sub(r"([a-z])([A-Z])", r"\1_\2", key).upper()

BIN
example_blueprint.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

BIN
example_dark.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 62 KiB

BIN
example_normal.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 60 KiB

BIN
example_transparent.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

125
format.md Normal file
View File

@ -0,0 +1,125 @@
# Schema Format
_**Supported syntaxes: JSON, XML, YAML**_
The following description uses the JSON syntax
For examples in different formats, see [example1.yaml](example1.yaml), [example2.yaml](example2.yaml), [example3.json](example3.json) and [example4.xml](example4.xml).
## Main layout
A schema contains a dictionary of structures. There must be at least one defined structure named "main"
```json
{
"structures": {
"main": {
...
},
"struct1": {
...
},
"struct2": {
...
},
...
}
}
```
## Structure
A structure has a given number of bits and one or multiple ranges. Each range of bits can have a name, a description and / or values with special meaning (see [Range](#range)). A range's structure can also depend on another range's value (see [Dependencies](#dependencies))
The range name (or key) defines the left- and rightmost bits (e.g. 7-4 goes from bit 7 to bit 4). Bits are displayed in big-endian, i.e. the leftmost bit has the highest value.
```json
"main": {
"bits": 8,
"ranges": {
"7-4": {
...
},
"3-2": {
...
},
"1": {
...
},
"0": {
...
}
}
}
```
## Range
A range represents a group of consecutive bits. It can have a name (display in the bit cells), a description (displayed under the structure) and / or values.
For values depending on other ranges, see [Dependencies](#dependencies).
> **Note**<br>
> In YAML, make sure to wrap values in quotes because some values can be interpreted as octal notation (e.g. 010)
```json
"3-2": {
"name": "op",
"description": "Logical operation",
"values": {
"00": "AND",
"01": "OR",
"10": "XOR",
"11": "NAND"
}
}
```
## Dependencies
The structure of one range may depend on the value of another. To represent this situation, first indicate on the child range the range on which it depends:
```json
"7-4": {
...
"depends-on": "0"
}
```
Then, in its values, indicate which structure to use. A description can also be added (displayed above the horizontal dependency arrow)
```json
"7-4": {
...
"depends-on": "0",
"values": {
"0": {
"description": "immediate value",
"structure": "immediateValue"
},
"1": {
"description": "value in register",
"structure": "registerValue"
}
}
}
```
Finally, add the sub-structures to the structure dictionary:
```json
{
"structures": {
"main": {
...
},
"immediateValue": {
"bits": 4,
...
},
"registerValue": {
"bits": 4,
...
},
...
}
}
```

46
main.py Normal file → Executable file
View File

@ -1,3 +1,8 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from __future__ import annotations
import argparse
import os
@ -15,20 +20,51 @@ description = """Examples:
- Transparent background:
python main.py schema.xml -o out.png -c transparent.json
- Directory mode:
python main.py -d -o images/ schemas/
"""
def processFile(inPath: str, outPath: str, confPath: str, display: bool) -> None:
schema = InstructionSetSchema(inPath, confPath, display)
schema.save(outPath)
if __name__ == "__main__":
parser = argparse.ArgumentParser(description=description, formatter_class=argparse.RawTextHelpFormatter)
parser.add_argument("schema", help="Path to the schema description. Accepted formats are: YAML, JSON and XML")
parser.add_argument("-o", "--output", help="Output path. By default, the output file will have the same name as the schema description with the extension .png")
parser.add_argument("-c", "--config", help="Path to the config file", default="config.json")
parser.add_argument("-D", "--display", help="Enable pygame display of the result", action="store_true")
parser.add_argument("-d", "--directory", help="Enable directory mode. If set, the input and output paths are directories and all files inside the input directory are processed", action="store_true")
args = parser.parse_args()
output = args.output
if output is None:
output = os.path.splitext(args.schema)[0] + ".png"
if args.directory:
if not os.path.isdir(args.schema):
print(f"{args.schema} is not a directory")
exit(-1)
output = args.output
if output is None:
output = args.schema
if not os.path.isdir(output):
print(f"{output} is not a directory")
exit(-1)
paths = os.listdir(args.schema)
for path in paths:
inPath = os.path.join(args.schema, path)
outPath = os.path.join(output, path)
outPath = os.path.splitext(outPath)[0] + ".png"
processFile(inPath, outPath, args.config, args.display)
schema = InstructionSetSchema(args.schema, args.config, args.display)
schema.save(output)
else:
output = args.output
if output is None:
output = os.path.splitext(args.schema)[0] + ".png"
processFile(args.schema, output, args.config, args.display)

View File

@ -1,5 +1,7 @@
from __future__ import annotations
from typing import Union
from typing import Union, Optional
class Range:
def __init__(self,
@ -7,21 +9,23 @@ class Range:
end: int,
name: str,
description: str = "",
values: dict[str, Union[str, dict]] = None,
dependsOn: str = None) -> None:
values: Optional[dict[str, Union[str, dict]]] = None,
dependsOn: Optional[tuple[int, int]] = None) -> None:
self.start = start
self.end = end
self.name = name
self.description = description
self.values = values
self.dependsOn = dependsOn
self.start: int = start
self.end: int = end
self.name: str = name
self.description: str = description
self.values: Optional[dict[str, Union[str, dict]]] = values
self.dependsOn: Optional[tuple[int, int]] = dependsOn
self.lastValueY: int = -1
@property
def bits(self) -> int:
return self.end - self.start + 1
def load(start: int, end: int, data: dict):
@staticmethod
def load(start: int, end: int, data: dict) -> Range:
values = None
bits = end - start + 1
@ -42,9 +46,11 @@ class Range:
values,
dependsOn)
@staticmethod
def parseSpan(span: str) -> tuple[int, int]:
startEnd = span.split("-")
if len(startEnd) == 1: startEnd.append(startEnd[0])
if len(startEnd) == 1:
startEnd.append(startEnd[0])
start = int(startEnd[1])
end = int(startEnd[0])
return (start, end)
return (start, end)

View File

@ -1,37 +1,48 @@
from __future__ import annotations
import os
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from config import Config
from range import Range
from schema import InstructionSetSchema
import pygame
from structure import Structure
from vec import Vec
if TYPE_CHECKING:
from config import Config
from range import Range
from schema import InstructionSetSchema
class Renderer:
WIDTH = 1200
HEIGHT = 800
def __init__(self, config: Config, display: bool = False) -> None:
self.config = config
self.display = display
self.config: Config = config
self.display: bool = display
pygame.init()
if self.display:
self.win = pygame.display.set_mode([Renderer.WIDTH, Renderer.HEIGHT])
self.win: pygame.Surface = pygame.display.set_mode([self.config.WIDTH, self.config.HEIGHT])
self.surf = pygame.Surface([Renderer.WIDTH, Renderer.HEIGHT], pygame.SRCALPHA)
self.font = pygame.font.SysFont(self.config.DEFAULT_FONT_FAMILY, self.config.DEFAULT_FONT_SIZE)
self.italicFont = pygame.font.SysFont(self.config.ITALIC_FONT_FAMILY, self.config.ITALIC_FONT_SIZE, italic=True)
self.surf: pygame.Surface = pygame.Surface([self.config.WIDTH, self.config.HEIGHT], pygame.SRCALPHA)
self.font: pygame.font.Font = pygame.font.SysFont(
self.config.DEFAULT_FONT_FAMILY,
self.config.DEFAULT_FONT_SIZE)
self.italicFont: pygame.font.Font = pygame.font.SysFont(
self.config.ITALIC_FONT_FAMILY,
self.config.ITALIC_FONT_SIZE, italic=True)
self.margins = self.config.MARGINS
self.margins: list[int, int, int, int] = self.config.MARGINS
def render(self, schema: InstructionSetSchema) -> None:
self.surf.fill(self.config.BACKGROUND_COLOR)
self.drawStructure(schema.structures["main"], schema.structures, self.margins[3], self.margins[0])
mainStruct: Structure = schema.structures["main"]
ox = self.margins[3]
if self.config.LEFT_LABELS:
ox = self.config.WIDTH - self.margins[3] - mainStruct.bits * self.config.BIT_WIDTH
self.drawStructure(schema.structures["main"], schema.structures, ox, self.margins[0])
if self.display:
name = os.path.basename(schema.path)
@ -45,8 +56,7 @@ class Renderer:
def drawStructure(self,
struct: Structure,
structures: dict[str,
Structure],
structures: dict[str, Structure],
ox: float = 0,
oy: float = 0) -> float:
@ -77,7 +87,21 @@ class Renderer:
pygame.draw.line(self.surf, borderCol, [bitX, bitsY], [bitX, bitsY + bitH])
ranges = struct.getSortedRanges()
descX = ox + max(0, (struct.bits-12) * bitW)
if self.config.LEFT_LABELS:
ranges.reverse()
if self.config.FORCE_DESCS_ON_SIDE:
if self.config.LEFT_LABELS:
descX = self.config.WIDTH - self.margins[3] - structures["main"].bits * bitW
else:
descX = self.margins[3] + structures["main"].bits * bitW
else:
if self.config.LEFT_LABELS:
descX = ox + struct.bits * bitW
else:
descX = ox
descY = bitsY + bitH * 2
# Names + simple descriptions
@ -123,10 +147,16 @@ class Renderer:
bitH = self.config.BIT_HEIGHT
arrowMargin = self.config.ARROW_MARGIN
if endX > startX:
endX -= arrowMargin
else:
endX += arrowMargin
pygame.draw.lines(self.surf, self.config.LINK_COLOR, False, [
[startX, startY + bitH*1.5],
[startX, endY + bitH/2],
[endX - arrowMargin, endY + bitH/2]
[endX, endY + bitH/2]
])
def drawDescription(self,
@ -140,7 +170,10 @@ class Renderer:
bitW = self.config.BIT_WIDTH
bitH = self.config.BIT_HEIGHT
descX = max(descX, rStartX + rWidth/2 + bitW)
if self.config.LEFT_LABELS:
descX = min(descX, rStartX + rWidth/2 - bitW)
else:
descX = max(descX, rStartX + rWidth/2 + bitW)
self.drawUnderbracket(rStartX, rStartX + rWidth, rStartY)
@ -148,12 +181,17 @@ class Renderer:
self.drawLink(midX, rStartY, descX, descY)
descTxt = self.font.render(range_.description, True, self.config.TEXT_COLOR)
self.surf.blit(descTxt, [descX, descY + (bitH - descTxt.get_height())/2])
txtX = descX
if self.config.LEFT_LABELS:
txtX -= descTxt.get_width()
self.surf.blit(descTxt, [txtX, descY + (bitH - descTxt.get_height())/2])
descY += descTxt.get_height()
if range_.values is not None and range_.dependsOn is None:
descX, descY = self.drawValues(range_.values, descX, descY)
txtX, descY = self.drawValues(range_.values, txtX, descY)
descY += self.config.DESCRIPTION_MARGIN
@ -174,8 +212,7 @@ class Renderer:
def drawDependency(self,
struct: Structure,
structures: dict[str,
Structure],
structures: dict[str, Structure],
bitsX: float,
bitsY: float,
range_: Range,
@ -191,8 +228,13 @@ class Renderer:
rWidth = range_.bits * bitW
self.drawUnderbracket(rStartX, rStartX + rWidth, bitsY)
prevY = bitsY + bitH * 1.5
dependRange = struct.ranges[range_.dependsOn]
prevRangeY = bitsY + bitH * 1.5
if dependRange.lastValueY == -1:
prevDependY = bitsY + bitH * 1.5
else:
prevDependY = dependRange.lastValueY
dependStartI = struct.bits - dependRange.end - 1
dependStartX = bitsX + dependStartI * bitW
dependWidth = dependRange.bits * bitW
@ -200,7 +242,7 @@ class Renderer:
self.drawUnderbracket(dependStartX, dependStartX + dependWidth, bitsY)
for val, data in sorted(range_.values.items(), key=lambda vd: vd[0]):
self.drawArrow(dependMid, prevY, dependMid, descY - arrowMargin)
self.drawArrow(dependMid, prevDependY, dependMid, descY - arrowMargin)
valRanges = {}
for i in range(dependRange.bits):
@ -232,11 +274,13 @@ class Renderer:
self.drawArrow(x1, y, x2, y, data["description"])
self.drawArrow(rStartX + rWidth - bitW,
prevY,
prevRangeY,
rStartX + rWidth - bitW,
descY + bitH - arrowMargin)
prevY = descY + bitH*2 + arrowMargin
prevDependY = descY + bitH*2 + arrowMargin
prevRangeY = prevDependY
dependRange.lastValueY = prevDependY
descY = self.drawStructure(structures[data["structure"]], structures, rStartX, descY)
return (descX, descY)
@ -276,4 +320,4 @@ class Renderer:
self.surf.blit(txt, [
(start.x + end.x - txt.get_width())/2,
(start.y + end.y)/2 + arrowLabelDist
])
])

View File

@ -1,5 +1,6 @@
import json
import os
import yaml
from config import Config
@ -7,16 +8,19 @@ from renderer import Renderer
from structure import Structure
from xml_loader import XMLLoader
class UnsupportedFormatException(Exception):
...
class InstructionSetSchema:
VALID_EXTENSIONS = ("yaml", "json", "xml")
def __init__(self, path: str, configPath: str = "config.json", display: bool = False) -> None:
self.config = Config(configPath)
self.display = display
self.path = path
self.config: Config = Config(configPath)
self.display: bool = display
self.path: str = path
self.structures: dict[str, Structure] = {}
self.load()
def load(self) -> None:
@ -48,4 +52,4 @@ class InstructionSetSchema:
renderer.render(self)
renderer.save(path)
if self.display:
input("Press ENTER to quit")
input("Press ENTER to quit")

View File

@ -1,18 +1,21 @@
from __future__ import annotations
from range import Range
class Structure:
def __init__(self,
name: str,
bits: int,
ranges: dict[str, Range],
ranges: dict[tuple[int, int], Range],
start: int = 0) -> None:
self.name = name
self.bits = bits
self.ranges = ranges
self.start = start
self.name: str = name
self.bits: int = bits
self.ranges: dict[tuple[int, int], Range] = ranges
self.start: int = start
@staticmethod
def load(id_: str, data: dict) -> Structure:
ranges = {}
for rSpan, rData in data["ranges"].items():
@ -32,4 +35,4 @@ class Structure:
def getSortedRanges(self) -> list[Range]:
ranges = self.ranges.values()
return list(sorted(ranges, key=lambda r: r.end))
return list(sorted(ranges, key=lambda r: r.end))

11
vec.py
View File

@ -1,10 +1,12 @@
from __future__ import annotations
from math import sqrt
class Vec:
def __init__(self, x: float = 0, y: float = 0) -> None:
self.x = x
self.y = y
self.x: float = x
self.y: float = y
def __add__(self, v: Vec) -> Vec:
return Vec(self.x+v.x, self.y+v.y)
@ -23,5 +25,6 @@ class Vec:
def norm(self) -> Vec:
mag = self.mag()
if mag == 0: return Vec()
return self/mag
if mag == 0:
return Vec()
return self / mag

View File

@ -1,11 +1,15 @@
from __future__ import annotations
from bs4 import BeautifulSoup
from typing import TYPE_CHECKING
from bs4 import BeautifulSoup
if TYPE_CHECKING:
from io import TextIOWrapper
class XMLLoader:
@staticmethod
def load(file_: TextIOWrapper) -> dict:
schema = {}
bs = BeautifulSoup(file_.read(), "xml")
@ -17,10 +21,12 @@ class XMLLoader:
schema["structures"] = structures
return schema
@staticmethod
def parseStructure(structElmt: any) -> dict:
struct = {}
struct["bits"] = structElmt.get("bits")
struct = {
"bits": structElmt.get("bits")
}
ranges = {}
rangeElmts = structElmt.findAll("range")
for rangeElmt in rangeElmts:
@ -30,11 +36,14 @@ class XMLLoader:
struct["ranges"] = ranges
return struct
@staticmethod
def parseRange(rangeElmt: any) -> dict:
range_ = {}
range_["name"] = rangeElmt.get("name")
range_ = {
"name": rangeElmt.get("name")
}
desc = rangeElmt.find("description")
if desc is not None: range_["description"] = desc.getText()
if desc is not None:
range_["description"] = desc.getText()
valuesElmt = rangeElmt.find("values")
if valuesElmt is not None:
@ -45,6 +54,7 @@ class XMLLoader:
return range_
@staticmethod
def parseValues(valuesElmt: any) -> dict:
values = {}
caseElmts = valuesElmt.findAll("case")
@ -62,4 +72,4 @@ class XMLLoader:
else:
values[val] = desc
return values
return values