fixed + completed type hints

.gitignore

@@ -1,2 +1,3 @@
 __pycache__
 *.jpg
+*.png

README.md

@@ -0,0 +1,181 @@
+<h3>rivet - Register / Instruction Visualizer and Explainer Tool</h3>
+
+---
+## Table of contents
+<!-- TOC -->
+  * [Table of contents](#table-of-contents)
+  * [Introduction](#introduction)
+  * [Requirements](#requirements)
+  * [Usage](#usage)
+  * [Options](#options)
+  * [Examples](#examples)
+      * [config.json](#configjson)
+      * [dark.json](#darkjson)
+      * [blueprint.json](#blueprintjson)
+      * [transparent.json](#transparentjson)
+  * [Config](#config)
+<!-- TOC -->
+
+---
+
+## Introduction
+
+This tool lets you generate visual explanations of binary instructions, or describe the contents of a register.
+
+The layout and style can be customized to fit your needs.
+
+For a full description of the schema format, please check out [format.md](format.md)
+
+## Requirements
+
+- [Python 3+](https://www.python.org/)
+- [Pygame](https://pypi.org/project/pygame/) - used to render the images
+- [Beautiful Soup 4](https://pypi.org/project/beautifulsoup4/) - used to parse XML files
+- [PyYAML](https://pypi.org/project/PyYAML/) - used to parse YAML files
+
+## Usage
+
+Basic usage:
+```bash
+python3 main.py schema.xml
+```
+
+Directory mode + config:
+```bash
+python3 main.py -o out/ -c config.json -d schemas/
+```
+
+## Options
+
+Several command line options are available:
+- `-o PATH` sets the output path. If not given, the output file will be located in the same directory as the input file, and given the same name (different extension)
+- `-d` enables directory mode. If set, the input and output paths are directories and all files inside the input directory are processed
+- `-c PATH` sets the config path. For more details on available config parameters, please read the [Config section](#config).
+
+## Examples
+
+The following images were generated from this schema ([example1.yaml](example1.yaml)):
+
+<details>
+<summary>Show / hide</summary>
+
+```yaml
+structures:
+  main:
+    bits: 32
+    ranges:
+      31-28:
+        name: cond
+      27:
+        name: 0
+      26:
+        name: 1
+      25:
+        name: I
+      24:
+        name: P
+        description: pre / post indexing bit
+        values:
+          0: post, add offset after transfer
+          1: pre, add offset before transfer
+      23:
+        name: U
+        description: up / down bit
+        values:
+          0: down, subtract offset from base
+          1: up, addition offset to base
+      22:
+        name: B
+        description: byte / word bit
+        values:
+          0: transfer word quantity
+          1: transfer byte quantity
+      21:
+        name: W
+        description: write-back bit
+        values:
+          0: no write-back
+          1: write address into base
+      20:
+        name: L
+        description: load / store bit
+        values:
+          0: store to memory
+          1: load from memory
+      19-16:
+        name: Rn
+        description: base register
+      15-12:
+        name: Rd
+        description: source / destination register
+      11-0:
+        name: offset
+        depends-on: 25
+        values:
+          0:
+            description: offset is an immediate value
+            structure: immediateOffset
+          1:
+            description: offset is a register
+            structure: registerOffset
+  immediateOffset:
+    bits: 12
+    ranges:
+      11-0:
+        name: 12-bit immediate offset
+        description: unsigned number
+  registerOffset:
+    bits: 12
+    ranges:
+      11-4:
+        name: shift
+        description: shift applied to Rm
+      3-0:
+        name: Rm
+        description: offset register
+```
+</details>
+
+#### config.json
+![default black on white](example_normal.png)
+
+#### dark.json
+![white on black](example_dark.png)
+
+#### blueprint.json
+![white on blue](example_blueprint.png)
+
+#### transparent.json
+![grey on transparent](example_transparent.png)
+
+## Config
+
+The config file may change the following values to customize the layout and style:
+- `defaultFontFamily`: the default font family
+- `defaultFontSize`: the default font size
+- `italicFontFamily`: the italic font family (for value description)
+- `italicFontSize`: the italic font size
+- `backgroundColor`: the image background color (ex: [222, 250, 206])
+- `textColor`: the default text color
+- `linkColor`: the color of linking lines and arrows
+- `borderColor`: the color of register borders
+- `bitWidth`: the width of 1 bit (in pixels)
+- `bitHeight`: the height of 1 bit (in pixels)
+- `descriptionMargin`: the space between descriptions (in pixels)
+- `dashLength`: the length of one dash (for dashed lines)
+- `dashSpace`: the space between dashes (for dashed lines)
+- `arrowSize`: the arrow size (height in pixels, width for horizontal arrow)
+- `margins`: the margins from the borders of the image (in pixels, [top, right, bottom, left])
+- `arrowMargin`: the margin between arrows and registers (in pixels)
+- `valuesGap`: the gap between values (in pixels)
+- `arrowLabelDistance`: the distance between arrows and their label (in pixels)
+- `forceDescsOnSide`: if true, descriptions are placed on the side of the register, otherwise, they are placed as close as possible to the bit
+- `leftLabels`: if true, descriptions are put on the left, otherwise, they default to the right hand side
+- `width`: the image width (in pixels)
+- `height`: the image height (in pixels)
+
+Some config examples are already included:
+- [config.json](config.json): Default configuration values (black on white)
+- [dark.json](dark.json): Dark theme (white on black)
+- [blueprint.json](blueprint.json): Blueprint theme (white on blue)
+- [transparent.json](transparent.json): Transparent background, for easy integration in other documents (grey on transparent)

config.json

@@ -17,5 +17,9 @@
     "margins": [20, 20, 20, 20],
     "arrowMargin": 4,
     "valuesGap": 5,
-    "arrowLabelDistance": 5
+    "arrowLabelDistance": 5,
+    "forceDescsOnSide": false,
+    "leftLabels": false,
+    "width": 1200,
+    "height": 800
 }

config.py

@@ -1,6 +1,9 @@
+from __future__ import annotations
+
 import json
 import re
 
+
 class Config:
     DEFAULT_FONT_FAMILY = "Ubuntu Mono"
     DEFAULT_FONT_SIZE = 16
@@ -21,11 +24,16 @@ class Config:
     ARROW_MARGIN = 4
     VALUES_GAP = 5
     ARROW_LABEL_DISTANCE = 5
+    FORCE_DESCS_ON_SIDE = False
+    LEFT_LABELS = False
+    WIDTH = 1200
+    HEIGHT = 800
     
     def __init__(self, path: str = "config.json") -> None:
         self.load(path)
 
-    def load(self, path: str) -> None:
+    @staticmethod
+    def load(path: str) -> None:
         with open(path, "r") as f:
             config = json.load(f)
             
@@ -34,5 +42,6 @@ class Config:
                 if hasattr(Config, k):
                     setattr(Config, k, v)
 
+    @staticmethod
     def formatKey(key: str) -> str:
         return re.sub(r"([a-z])([A-Z])", r"\1_\2", key).upper()

format.md

@@ -0,0 +1,125 @@
+# Schema Format
+
+_**Supported syntaxes: JSON, XML, YAML**_
+
+The following description uses the JSON syntax
+
+For examples in different formats, see [example1.yaml](example1.yaml), [example2.yaml](example2.yaml), [example3.json](example3.json) and [example4.xml](example4.xml).
+
+## Main layout
+
+A schema contains a dictionary of structures. There must be at least one defined structure named "main" 
+```json
+{
+  "structures": {
+    "main": {
+      ...
+    },
+    "struct1": {
+      ...
+    },
+    "struct2": {
+      ...
+    },
+    ...
+  }
+}
+```
+
+## Structure
+
+A structure has a given number of bits and one or multiple ranges. Each range of bits can have a name, a description and / or values with special meaning (see [Range](#range)). A range's structure can also depend on another range's value (see [Dependencies](#dependencies))
+
+The range name (or key) defines the left- and rightmost bits (e.g. 7-4 goes from bit 7 to bit 4). Bits are displayed in big-endian, i.e. the leftmost bit has the highest value.
+```json
+"main": {
+  "bits": 8,
+  "ranges": {
+    "7-4": {
+      ...
+    },
+    "3-2": {
+      ...
+    },
+    "1": {
+      ...
+    },
+    "0": {
+      ...
+    }
+  }
+}
+```
+
+## Range
+
+A range represents a group of consecutive bits. It can have a name (display in the bit cells), a description (displayed under the structure) and / or values.
+
+For values depending on other ranges, see [Dependencies](#dependencies).
+
+> **Note**<br>
+> In YAML, make sure to wrap values in quotes because some values can be interpreted as octal notation (e.g. 010)
+
+```json
+"3-2": {
+  "name": "op",
+  "description": "Logical operation",
+  "values": {
+    "00": "AND",
+    "01": "OR",
+    "10": "XOR",
+    "11": "NAND"
+  }
+}
+```
+
+## Dependencies
+
+The structure of one range may depend on the value of another. To represent this situation, first indicate on the child range the range on which it depends:
+
+```json
+"7-4": {
+  ...
+  "depends-on": "0"
+}
+```
+
+Then, in its values, indicate which structure to use. A description can also be added (displayed above the horizontal dependency arrow)
+
+```json
+"7-4": {
+  ...
+  "depends-on": "0",
+  "values": {
+    "0": {
+      "description": "immediate value",
+      "structure": "immediateValue"
+    },
+    "1": {
+      "description": "value in register",
+      "structure": "registerValue"
+    }
+  }
+}
+```
+
+Finally, add the sub-structures to the structure dictionary:
+
+```json
+{
+  "structures": {
+    "main": {
+      ...
+    },
+    "immediateValue": {
+      "bits": 4,
+      ...
+    },
+    "registerValue": {
+      "bits": 4,
+      ...
+    },
+    ...
+  }
+}
+```

main.py

@@ -1,3 +1,8 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+from __future__ import annotations
+
 import argparse
 import os
 
@@ -15,20 +20,51 @@ description = """Examples:
 
     - Transparent background:
         python main.py schema.xml -o out.png -c transparent.json
+    
+    - Directory mode:
+        python main.py -d -o images/ schemas/
 """
 
+
+def processFile(inPath: str, outPath: str, confPath: str, display: bool) -> None:
+    schema = InstructionSetSchema(inPath, confPath, display)
+    schema.save(outPath)
+
+
 if __name__ == "__main__":
     parser = argparse.ArgumentParser(description=description, formatter_class=argparse.RawTextHelpFormatter)
     parser.add_argument("schema", help="Path to the schema description. Accepted formats are: YAML, JSON and XML")
     parser.add_argument("-o", "--output", help="Output path. By default, the output file will have the same name as the schema description with the extension .png")
     parser.add_argument("-c", "--config", help="Path to the config file", default="config.json")
     parser.add_argument("-D", "--display", help="Enable pygame display of the result", action="store_true")
+    parser.add_argument("-d", "--directory", help="Enable directory mode. If set, the input and output paths are directories and all files inside the input directory are processed", action="store_true")
     
     args = parser.parse_args()
     
+    if args.directory:
+        if not os.path.isdir(args.schema):
+            print(f"{args.schema} is not a directory")
+            exit(-1)
+        
+        output = args.output
+        if output is None:
+            output = args.schema
+        
+        if not os.path.isdir(output):
+            print(f"{output} is not a directory")
+            exit(-1)
+        
+        paths = os.listdir(args.schema)
+        for path in paths:
+            inPath = os.path.join(args.schema, path)
+            outPath = os.path.join(output, path)
+            outPath = os.path.splitext(outPath)[0] + ".png"
+            
+            processFile(inPath, outPath, args.config, args.display)
+    
+    else:
         output = args.output
         if output is None:
             output = os.path.splitext(args.schema)[0] + ".png"
         
-    schema = InstructionSetSchema(args.schema, args.config, args.display)
+        processFile(args.schema, output, args.config, args.display)
-    schema.save(output)

range.py

@@ -1,5 +1,7 @@
 from __future__ import annotations
-from typing import Union
+
+from typing import Union, Optional
+
 
 class Range:
     def __init__(self,
@@ -7,21 +9,23 @@ class Range:
                  end: int,
                  name: str,
                  description: str = "",
-                 values: dict[str, Union[str, dict]] = None,
+                 values: Optional[dict[str, Union[str, dict]]] = None,
-                 dependsOn: str = None) -> None:
+                 dependsOn: Optional[tuple[int, int]] = None) -> None:
         
-        self.start = start
+        self.start: int = start
-        self.end = end
+        self.end: int = end
-        self.name = name
+        self.name: str = name
-        self.description = description
+        self.description: str = description
-        self.values = values
+        self.values: Optional[dict[str, Union[str, dict]]] = values
-        self.dependsOn = dependsOn
+        self.dependsOn: Optional[tuple[int, int]] = dependsOn
+        self.lastValueY: int = -1
     
     @property
     def bits(self) -> int:
         return self.end - self.start + 1
 
-    def load(start: int, end: int, data: dict):
+    @staticmethod
+    def load(start: int, end: int, data: dict) -> Range:
         values = None
         bits = end - start + 1
         
@@ -42,9 +46,11 @@ class Range:
                      values,
                      dependsOn)
 
+    @staticmethod
     def parseSpan(span: str) -> tuple[int, int]:
         startEnd = span.split("-")
-        if len(startEnd) == 1: startEnd.append(startEnd[0])
+        if len(startEnd) == 1:
+            startEnd.append(startEnd[0])
         start = int(startEnd[1])
         end = int(startEnd[0])
         return (start, end)

renderer.py

@@ -1,37 +1,48 @@
 from __future__ import annotations
+
 import os
 from typing import TYPE_CHECKING
-if TYPE_CHECKING:
-    from config import Config
-    from range import Range
-    from schema import InstructionSetSchema
 
 import pygame
 
 from structure import Structure
 from vec import Vec
 
-class Renderer:
+if TYPE_CHECKING:
-    WIDTH = 1200
+    from config import Config
-    HEIGHT = 800
+    from range import Range
+    from schema import InstructionSetSchema
 
+
+class Renderer:
     def __init__(self, config: Config, display: bool = False) -> None:
-        self.config = config
+        self.config: Config = config
-        self.display = display
+        self.display: bool = display
         pygame.init()
         if self.display:
-            self.win = pygame.display.set_mode([Renderer.WIDTH, Renderer.HEIGHT])
+            self.win: pygame.Surface = pygame.display.set_mode([self.config.WIDTH, self.config.HEIGHT])
         
-        self.surf = pygame.Surface([Renderer.WIDTH, Renderer.HEIGHT], pygame.SRCALPHA)
+        self.surf: pygame.Surface = pygame.Surface([self.config.WIDTH, self.config.HEIGHT], pygame.SRCALPHA)
-        self.font = pygame.font.SysFont(self.config.DEFAULT_FONT_FAMILY, self.config.DEFAULT_FONT_SIZE)
-        self.italicFont = pygame.font.SysFont(self.config.ITALIC_FONT_FAMILY, self.config.ITALIC_FONT_SIZE, italic=True)
 
-        self.margins = self.config.MARGINS
+        self.font: pygame.font.Font = pygame.font.SysFont(
+            self.config.DEFAULT_FONT_FAMILY,
+            self.config.DEFAULT_FONT_SIZE)
+
+        self.italicFont: pygame.font.Font = pygame.font.SysFont(
+            self.config.ITALIC_FONT_FAMILY,
+            self.config.ITALIC_FONT_SIZE, italic=True)
+        
+        self.margins: list[int, int, int, int] = self.config.MARGINS
     
     def render(self, schema: InstructionSetSchema) -> None:
         self.surf.fill(self.config.BACKGROUND_COLOR)
         
-        self.drawStructure(schema.structures["main"], schema.structures, self.margins[3], self.margins[0])
+        mainStruct: Structure = schema.structures["main"]
+        ox = self.margins[3]
+        if self.config.LEFT_LABELS:
+            ox = self.config.WIDTH - self.margins[3] - mainStruct.bits * self.config.BIT_WIDTH
+        
+        self.drawStructure(schema.structures["main"], schema.structures, ox, self.margins[0])
         
         if self.display:
             name = os.path.basename(schema.path)
@@ -45,8 +56,7 @@ class Renderer:
     
     def drawStructure(self,
                       struct: Structure,
-                      structures: dict[str,
+                      structures: dict[str, Structure],
-                      Structure],
                       ox: float = 0,
                       oy: float = 0) -> float:
         
@@ -77,7 +87,21 @@ class Renderer:
                 pygame.draw.line(self.surf, borderCol, [bitX, bitsY], [bitX, bitsY + bitH])
         
         ranges = struct.getSortedRanges()
-        descX = ox + max(0, (struct.bits-12) * bitW)
+        if self.config.LEFT_LABELS:
+            ranges.reverse()
+        
+        if self.config.FORCE_DESCS_ON_SIDE:
+            if self.config.LEFT_LABELS:
+                descX = self.config.WIDTH - self.margins[3] - structures["main"].bits * bitW
+            else:
+                descX = self.margins[3] + structures["main"].bits * bitW
+        
+        else:
+            if self.config.LEFT_LABELS:
+                descX = ox + struct.bits * bitW
+            else:
+                descX = ox
+        
         descY = bitsY + bitH * 2
         
         # Names + simple descriptions
@@ -123,10 +147,16 @@ class Renderer:
         bitH = self.config.BIT_HEIGHT
         arrowMargin = self.config.ARROW_MARGIN
         
+        if endX > startX:
+            endX -= arrowMargin
+        
+        else:
+            endX += arrowMargin
+        
         pygame.draw.lines(self.surf, self.config.LINK_COLOR, False, [
             [startX, startY + bitH*1.5],
             [startX, endY + bitH/2],
-            [endX - arrowMargin, endY + bitH/2]
+            [endX, endY + bitH/2]
         ])
     
     def drawDescription(self,
@@ -140,6 +170,9 @@ class Renderer:
         bitW = self.config.BIT_WIDTH
         bitH = self.config.BIT_HEIGHT
         
+        if self.config.LEFT_LABELS:
+            descX = min(descX, rStartX + rWidth/2 - bitW)
+        else:
             descX = max(descX, rStartX + rWidth/2 + bitW)
         
         self.drawUnderbracket(rStartX, rStartX + rWidth, rStartY)
@@ -148,12 +181,17 @@ class Renderer:
         self.drawLink(midX, rStartY, descX, descY)
         
         descTxt = self.font.render(range_.description, True, self.config.TEXT_COLOR)
-        self.surf.blit(descTxt, [descX, descY + (bitH - descTxt.get_height())/2])
+        txtX = descX
+        
+        if self.config.LEFT_LABELS:
+            txtX -= descTxt.get_width()
+        
+        self.surf.blit(descTxt, [txtX, descY + (bitH - descTxt.get_height())/2])
         
         descY += descTxt.get_height()
         
         if range_.values is not None and range_.dependsOn is None:
-            descX, descY = self.drawValues(range_.values, descX, descY)
+            txtX, descY = self.drawValues(range_.values, txtX, descY)
         
         descY += self.config.DESCRIPTION_MARGIN
         
@@ -174,8 +212,7 @@ class Renderer:
 
     def drawDependency(self,
                        struct: Structure,
-                       structures: dict[str,
+                       structures: dict[str, Structure],
-                       Structure],
                        bitsX: float,
                        bitsY: float,
                        range_: Range,
@@ -191,8 +228,13 @@ class Renderer:
         rWidth = range_.bits * bitW
         
         self.drawUnderbracket(rStartX, rStartX + rWidth, bitsY)
-        prevY = bitsY + bitH * 1.5
         dependRange = struct.ranges[range_.dependsOn]
+        prevRangeY = bitsY + bitH * 1.5
+        if dependRange.lastValueY == -1:
+            prevDependY = bitsY + bitH * 1.5
+        else:
+            prevDependY = dependRange.lastValueY
+        
         dependStartI = struct.bits - dependRange.end - 1
         dependStartX = bitsX + dependStartI * bitW
         dependWidth = dependRange.bits * bitW
@@ -200,7 +242,7 @@ class Renderer:
         self.drawUnderbracket(dependStartX, dependStartX + dependWidth, bitsY)
         
         for val, data in sorted(range_.values.items(), key=lambda vd: vd[0]):
-            self.drawArrow(dependMid, prevY, dependMid, descY - arrowMargin)
+            self.drawArrow(dependMid, prevDependY, dependMid, descY - arrowMargin)
             
             valRanges = {}
             for i in range(dependRange.bits):
@@ -232,11 +274,13 @@ class Renderer:
             self.drawArrow(x1, y, x2, y, data["description"])
             
             self.drawArrow(rStartX + rWidth - bitW,
-                           prevY,
+                           prevRangeY,
                            rStartX + rWidth - bitW,
                            descY + bitH - arrowMargin)
 
-            prevY = descY + bitH*2 + arrowMargin
+            prevDependY = descY + bitH*2 + arrowMargin
+            prevRangeY = prevDependY
+            dependRange.lastValueY = prevDependY
             descY = self.drawStructure(structures[data["structure"]], structures, rStartX, descY)
         
         return (descX, descY)

schema.py

@@ -1,5 +1,6 @@
 import json
 import os
+
 import yaml
 
 from config import Config
@@ -7,16 +8,19 @@ from renderer import Renderer
 from structure import Structure
 from xml_loader import XMLLoader
 
+
 class UnsupportedFormatException(Exception):
     ...
 
+
 class InstructionSetSchema:
     VALID_EXTENSIONS = ("yaml", "json", "xml")
     
     def __init__(self, path: str, configPath: str = "config.json", display: bool = False) -> None:
-        self.config = Config(configPath)
+        self.config: Config = Config(configPath)
-        self.display = display
+        self.display: bool = display
-        self.path = path
+        self.path: str = path
+        self.structures: dict[str, Structure] = {}
         self.load()
     
     def load(self) -> None:

structure.py

@@ -1,18 +1,21 @@
 from __future__ import annotations
+
 from range import Range
 
+
 class Structure:
     def __init__(self,
                  name: str,
                  bits: int,
-                 ranges: dict[str, Range],
+                 ranges: dict[tuple[int, int], Range],
                  start: int = 0) -> None:
         
-        self.name = name
+        self.name: str = name
-        self.bits = bits
+        self.bits: int = bits
-        self.ranges = ranges
+        self.ranges: dict[tuple[int, int], Range] = ranges
-        self.start = start
+        self.start: int = start
 
+    @staticmethod
     def load(id_: str, data: dict) -> Structure:
         ranges = {}
         for rSpan, rData in data["ranges"].items():

vec.py

@@ -1,10 +1,12 @@
 from __future__ import annotations
+
 from math import sqrt
 
+
 class Vec:
     def __init__(self, x: float = 0, y: float = 0) -> None:
-        self.x = x
+        self.x: float = x
-        self.y = y
+        self.y: float = y
     
     def __add__(self, v: Vec) -> Vec:
         return Vec(self.x+v.x, self.y+v.y)
@@ -23,5 +25,6 @@ class Vec:
     
     def norm(self) -> Vec:
         mag = self.mag()
-        if mag == 0: return Vec()
+        if mag == 0:
+            return Vec()
         return self / mag

xml_loader.py

@@ -1,11 +1,15 @@
 from __future__ import annotations
-from bs4 import BeautifulSoup
+
 from typing import TYPE_CHECKING
 
+from bs4 import BeautifulSoup
+
 if TYPE_CHECKING:
     from io import TextIOWrapper
 
+
 class XMLLoader:
+    @staticmethod
     def load(file_: TextIOWrapper) -> dict:
         schema = {}
         bs = BeautifulSoup(file_.read(), "xml")
@@ -18,9 +22,11 @@ class XMLLoader:
         schema["structures"] = structures
         return schema
 
+    @staticmethod
     def parseStructure(structElmt: any) -> dict:
-        struct = {}
+        struct = {
-        struct["bits"] = structElmt.get("bits")
+            "bits": structElmt.get("bits")
+        }
         ranges = {}
         rangeElmts = structElmt.findAll("range")
         for rangeElmt in rangeElmts:
@@ -30,11 +36,14 @@ class XMLLoader:
         struct["ranges"] = ranges
         return struct
 
+    @staticmethod
     def parseRange(rangeElmt: any) -> dict:
-        range_ = {}
+        range_ = {
-        range_["name"] = rangeElmt.get("name")
+            "name": rangeElmt.get("name")
+        }
         desc = rangeElmt.find("description")
-        if desc is not None: range_["description"] = desc.getText()
+        if desc is not None:
+            range_["description"] = desc.getText()
         
         valuesElmt = rangeElmt.find("values")
         if valuesElmt is not None:
@@ -45,6 +54,7 @@ class XMLLoader:
         
         return range_
 
+    @staticmethod
     def parseValues(valuesElmt: any) -> dict:
         values = {}
         caseElmts = valuesElmt.findAll("case")