0.0.5: new normalization class

Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    unisec (0.0.4)
+    unisec (0.0.5)
       ctf-party (~> 3.0)
       dry-cli (~> 1.0)
       paint (~> 2.3)
@@ -16,28 +16,28 @@ GEM
     cldr-plurals-runtime-rb (1.1.0)
     coderay (1.1.3)
     commonmarker (0.23.10)
-    concurrent-ruby (1.2.2)
+    concurrent-ruby (1.2.3)
     ctf-party (3.0.0)
       docopt (~> 0.6)
       uri (>= 0.12.1, < 0.14.0)
     docopt (0.6.1)
     dry-cli (1.0.0)
     json (2.7.1)
     language_server-protocol (3.17.0.3)
-    minitest (5.21.2)
+    minitest (5.22.2)
     minitest-skip (0.0.3)
       minitest (~> 5.0)
     paint (2.3.0)
     parallel (1.24.0)
-    parser (3.3.0.4)
+    parser (3.3.0.5)
       ast (~> 2.4.1)
       racc
     racc (1.7.3)
     rainbow (3.1.1)
     rake (13.1.0)
     regexp_parser (2.9.0)
     rexml (3.2.6)
-    rubocop (1.60.1)
+    rubocop (1.60.2)
       json (~> 2.3)
       language_server-protocol (>= 3.17.0)
       parallel (~> 1.10)

README.md

@@ -25,6 +25,8 @@ A CLI tool and library to play with Unicode security.
   - Replace all characters from a string with random confusables
 - **Hexdump**
   - UTF-8, UTF-16, UTF-32 hexadecimal dumps
+- **Normalization**
+  - NFC, NFKC, NFD, NFKD normalization forms
 - **Properties**
   - Get all properties of a given Unicode character
   - List code points matching a Unicode property

docs/CHANGELOG.md

@@ -1,5 +1,16 @@
 ## [unreleased]
 
+## [0.0.5]
+
+**Features**
+
+- Add a new class `Unisec::Normalization` and CLI command `normalize` to output all normalization forms
+
+**Chore**
+
+- Enhance documentation
+- Dependencies update
+
 ## [0.0.4]
 
 **Features**

docs/pages/publishing.md

@@ -12,6 +12,14 @@ Also check the **linter**:
 $ bundle exec rubocop
 ```
 
+Update the version in `lib/unisec/version.rb`.
+
+Update the documentation, at least:
+
+- `README.md`
+- `docs/CHANGELOG.md`
+- `docs/pages/usage.md`
+
 On new release don't forget to rebuild the **library documentation**:
 
 ```

docs/pages/usage.md

@@ -58,6 +58,7 @@ Options:
   - [Randomize](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Confusables/Randomize)
 - [Grep](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Grep)
 - [Hexdump](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Hexdump)
+- [Normalize](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Normalize)
 - **Properties**
   - [Char](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Properties/Char)
   - [Codepoints](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Properties/Codepoints)
@@ -77,6 +78,7 @@ See examples in [the library documentation](https://acceis.github.io/unisec/yard
 - [Unisec::Bidi](https://acceis.github.io/unisec/yard/Unisec/Bidi)
 - [Unisec::Confusables](https://acceis.github.io/unisec/yard/Unisec/Confusables)
 - [Unisec::Hexdump](https://acceis.github.io/unisec/yard/Unisec/Hexdump)
+- [Unisec::Normalization](https://acceis.github.io/unisec/yard/Unisec/Normalization)
 - [Unisec::Properties](https://acceis.github.io/unisec/yard/Unisec/Properties)
 - [Unisec::Rugrep](https://acceis.github.io/unisec/yard/Unisec/Rugrep)
 - [Unisec::Size](https://acceis.github.io/unisec/yard/Unisec/Size)

docs/yard/Integer.html

@@ -304,7 +304,7 @@ <h3 class="signature " id="to_hex-instance_method">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:47 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec.html

@@ -79,7 +79,7 @@
   <dl>
     <dt>Defined in:</dt>
     <dd>lib/unisec/bidi.rb<span class="defines">,<br />
-  lib/unisec/size.rb,<br /> lib/unisec/utils.rb,<br /> lib/unisec/rugrep.rb,<br /> lib/unisec/cli/cli.rb,<br /> lib/unisec/hexdump.rb,<br /> lib/unisec/version.rb,<br /> lib/unisec/cli/bidi.rb,<br /> lib/unisec/cli/size.rb,<br /> lib/unisec/versions.rb,<br /> lib/unisec/cli/rugrep.rb,<br /> lib/unisec/properties.rb,<br /> lib/unisec/surrogates.rb,<br /> lib/unisec/cli/hexdump.rb,<br /> lib/unisec/confusables.rb,<br /> lib/unisec/cli/versions.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb,<br /> lib/unisec/cli/confusables.rb</span>
+  lib/unisec/size.rb,<br /> lib/unisec/utils.rb,<br /> lib/unisec/rugrep.rb,<br /> lib/unisec/cli/cli.rb,<br /> lib/unisec/hexdump.rb,<br /> lib/unisec/version.rb,<br /> lib/unisec/cli/bidi.rb,<br /> lib/unisec/cli/size.rb,<br /> lib/unisec/versions.rb,<br /> lib/unisec/cli/rugrep.rb,<br /> lib/unisec/properties.rb,<br /> lib/unisec/surrogates.rb,<br /> lib/unisec/cli/hexdump.rb,<br /> lib/unisec/confusables.rb,<br /> lib/unisec/cli/versions.rb,<br /> lib/unisec/normalization.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb,<br /> lib/unisec/cli/confusables.rb,<br /> lib/unisec/cli/normalization.rb</span>
 </dd>
   </dl>
 
@@ -93,7 +93,7 @@ <h2>Defined Under Namespace</h2>
 
 
 
-      <strong class="classes">Classes:</strong> <span class='object_link'><a href="Unisec/Bidi.html" title="Unisec::Bidi (class)">Bidi</a></span>, <span class='object_link'><a href="Unisec/Confusables.html" title="Unisec::Confusables (class)">Confusables</a></span>, <span class='object_link'><a href="Unisec/Hexdump.html" title="Unisec::Hexdump (class)">Hexdump</a></span>, <span class='object_link'><a href="Unisec/Properties.html" title="Unisec::Properties (class)">Properties</a></span>, <span class='object_link'><a href="Unisec/Rugrep.html" title="Unisec::Rugrep (class)">Rugrep</a></span>, <span class='object_link'><a href="Unisec/Size.html" title="Unisec::Size (class)">Size</a></span>, <span class='object_link'><a href="Unisec/Surrogates.html" title="Unisec::Surrogates (class)">Surrogates</a></span>, <span class='object_link'><a href="Unisec/Versions.html" title="Unisec::Versions (class)">Versions</a></span>
+      <strong class="classes">Classes:</strong> <span class='object_link'><a href="Unisec/Bidi.html" title="Unisec::Bidi (class)">Bidi</a></span>, <span class='object_link'><a href="Unisec/Confusables.html" title="Unisec::Confusables (class)">Confusables</a></span>, <span class='object_link'><a href="Unisec/Hexdump.html" title="Unisec::Hexdump (class)">Hexdump</a></span>, <span class='object_link'><a href="Unisec/Normalization.html" title="Unisec::Normalization (class)">Normalization</a></span>, <span class='object_link'><a href="Unisec/Properties.html" title="Unisec::Properties (class)">Properties</a></span>, <span class='object_link'><a href="Unisec/Rugrep.html" title="Unisec::Rugrep (class)">Rugrep</a></span>, <span class='object_link'><a href="Unisec/Size.html" title="Unisec::Size (class)">Size</a></span>, <span class='object_link'><a href="Unisec/Surrogates.html" title="Unisec::Surrogates (class)">Surrogates</a></span>, <span class='object_link'><a href="Unisec/Versions.html" title="Unisec::Versions (class)">Versions</a></span>
 
 
 </p>
@@ -119,7 +119,7 @@ <h2>
 
 </div>
         </dt>
-        <dd><pre class="code"><span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">'</span><span style="color:#D20">0.0.4</span><span style="color:#710">'</span></span></pre></dd>
+        <dd><pre class="code"><span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">'</span><span style="color:#D20">0.0.5</span><span style="color:#710">'</span></span></pre></dd>
 
     </dl>
 
@@ -135,7 +135,7 @@ <h2>
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:46 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/Bidi.html

@@ -131,7 +131,7 @@ <h2>Overview</h2><div class="docstring">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:46 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/Bidi/Spoof.html

@@ -294,7 +294,7 @@ <h2>
         <li class="public ">
   <span class="summary_signature">
 
-      <a href="#display-instance_method" title="#display (instance method)">#<strong>display</strong>(light: false)  &#x21d2; Object </a>
+      <a href="#display-instance_method" title="#display (instance method)">#<strong>display</strong>(light: false)  &#x21d2; String </a>
 
 
 
@@ -1053,7 +1053,7 @@ <h3 class="signature first" id="bidi_affix-instance_method">
       <div class="method_details ">
   <h3 class="signature " id="display-instance_method">
 
-    #<strong>display</strong>(light: false)  &#x21d2; <tt>Object</tt> 
+    #<strong>display</strong>(light: false)  &#x21d2; <tt>String</tt> 
 
 
 
@@ -1114,14 +1114,30 @@ <h3 class="signature " id="display-instance_method">
 
 </ul>
 
+<p class="tag_title">Returns:</p>
+<ul class="return">
+
+    <li>
+
+
+        <span class='type'>(<tt>String</tt>)</span>
+
+
+
+        &mdash;
+        <div class='inline'><p>CLI-ready output</p>
+</div>
+
+    </li>
+
+</ul>
 
 </div><table class="source_code">
   <tr>
     <td>
       <pre class="lines">
 
 
-150
 151
 152
 153
@@ -1138,10 +1154,11 @@ <h3 class="signature " id="display-instance_method">
 164
 165
 166
-167</pre>
+167
+168</pre>
     </td>
     <td>
-      <pre class="code"><span class="info file"># File 'lib/unisec/bidi.rb', line 150</span>
+      <pre class="code"><span class="info file"># File 'lib/unisec/bidi.rb', line 151</span>
 
 <span style="color:#080;font-weight:bold">def</span> <span style="color:#06B;font-weight:bold">display</span>(<span style="color:#606">light</span>: <span style="color:#069">false</span>)
   <span style="color:#080;font-weight:bold">if</span> light == <span style="color:#069">false</span> <span style="color:#777"># full display</span>
@@ -1309,7 +1326,7 @@ <h3 class="signature " id="set_target_display-instance_method">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:46 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/CLI.html

@@ -79,7 +79,7 @@
   <dl>
     <dt>Defined in:</dt>
     <dd>lib/unisec/cli/cli.rb<span class="defines">,<br />
-  lib/unisec/cli/bidi.rb,<br /> lib/unisec/cli/size.rb,<br /> lib/unisec/cli/rugrep.rb,<br /> lib/unisec/cli/hexdump.rb,<br /> lib/unisec/cli/versions.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb,<br /> lib/unisec/cli/confusables.rb</span>
+  lib/unisec/cli/bidi.rb,<br /> lib/unisec/cli/size.rb,<br /> lib/unisec/cli/rugrep.rb,<br /> lib/unisec/cli/hexdump.rb,<br /> lib/unisec/cli/versions.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb,<br /> lib/unisec/cli/confusables.rb,<br /> lib/unisec/cli/normalization.rb</span>
 </dd>
   </dl>
 
@@ -117,7 +117,7 @@ <h2>Overview</h2><div class="docstring">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:46 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/CLI/Commands.html

@@ -84,7 +84,7 @@
   <dl>
     <dt>Defined in:</dt>
     <dd>lib/unisec/cli/cli.rb<span class="defines">,<br />
-  lib/unisec/cli/bidi.rb,<br /> lib/unisec/cli/size.rb,<br /> lib/unisec/cli/rugrep.rb,<br /> lib/unisec/cli/hexdump.rb,<br /> lib/unisec/cli/versions.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb,<br /> lib/unisec/cli/confusables.rb</span>
+  lib/unisec/cli/bidi.rb,<br /> lib/unisec/cli/size.rb,<br /> lib/unisec/cli/rugrep.rb,<br /> lib/unisec/cli/hexdump.rb,<br /> lib/unisec/cli/versions.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb,<br /> lib/unisec/cli/confusables.rb,<br /> lib/unisec/cli/normalization.rb</span>
 </dd>
   </dl>
 
@@ -108,7 +108,7 @@ <h2>Overview</h2><div class="docstring">
 
 
 
-      <strong class="classes">Classes:</strong> <span class='object_link'><a href="Commands/Grep.html" title="Unisec::CLI::Commands::Grep (class)">Grep</a></span>, <span class='object_link'><a href="Commands/Hexdump.html" title="Unisec::CLI::Commands::Hexdump (class)">Hexdump</a></span>, <span class='object_link'><a href="Commands/Size.html" title="Unisec::CLI::Commands::Size (class)">Size</a></span>, <span class='object_link'><a href="Commands/Versions.html" title="Unisec::CLI::Commands::Versions (class)">Versions</a></span>
+      <strong class="classes">Classes:</strong> <span class='object_link'><a href="Commands/Grep.html" title="Unisec::CLI::Commands::Grep (class)">Grep</a></span>, <span class='object_link'><a href="Commands/Hexdump.html" title="Unisec::CLI::Commands::Hexdump (class)">Hexdump</a></span>, <span class='object_link'><a href="Commands/Normalize.html" title="Unisec::CLI::Commands::Normalize (class)">Normalize</a></span>, <span class='object_link'><a href="Commands/Size.html" title="Unisec::CLI::Commands::Size (class)">Size</a></span>, <span class='object_link'><a href="Commands/Versions.html" title="Unisec::CLI::Commands::Versions (class)">Versions</a></span>
 
 
 </p>
@@ -125,7 +125,7 @@ <h2>Overview</h2><div class="docstring">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:46 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/CLI/Commands/Bidi.html

@@ -115,7 +115,7 @@ <h2>Overview</h2><div class="docstring">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:46 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/CLI/Commands/Bidi/Spoof.html

@@ -269,7 +269,7 @@ <h3 class="signature first" id="call-instance_method">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:47 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/CLI/Commands/Confusables.html

@@ -115,7 +115,7 @@ <h2>Overview</h2><div class="docstring">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:46 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/CLI/Commands/Confusables/List.html

@@ -272,7 +272,7 @@ <h3 class="signature first" id="call-instance_method">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:47 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/CLI/Commands/Confusables/Randomize.html

@@ -231,7 +231,7 @@ <h3 class="signature first" id="call-instance_method">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:47 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/CLI/Commands/Grep.html

@@ -231,7 +231,7 @@ <h3 class="signature first" id="call-instance_method">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:47 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>

docs/yard/Unisec/CLI/Commands/Hexdump.html

@@ -111,6 +111,9 @@ <h2>Overview</h2><div class="docstring">
 UTF-16LE: 4100 4300 4300 4500 4900 5300
 UTF-32BE: 00000041 00000043 00000043 00000045 00000049 00000053
 UTF-32LE: 41000000 43000000 43000000 45000000 49000000 53000000
+
+$unisec hexdump &quot;ACCEIS&quot; --enc utf16le
+4100 4300 4300 4500 4900 5300
 </code></pre>
 
 
@@ -213,26 +216,26 @@ <h3 class="signature first" id="call-instance_method">
       <pre class="lines">
 
 
-32
-33
-34
 35
 36
 37
 38
 39
-40</pre>
+40
+41
+42
+43</pre>
     </td>
     <td>
-      <pre class="code"><span class="info file"># File 'lib/unisec/cli/hexdump.rb', line 32</span>
+      <pre class="code"><span class="info file"># File 'lib/unisec/cli/hexdump.rb', line 35</span>
 
 <span style="color:#080;font-weight:bold">def</span> <span style="color:#06B;font-weight:bold">call</span>(<span style="color:#606">input</span>: <span style="color:#069">nil</span>, **options)
   input = <span style="color:#d70">$stdin</span>.read.chomp <span style="color:#080;font-weight:bold">if</span> input == <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">'</span><span style="color:#D20">-</span><span style="color:#710">'</span></span>
   <span style="color:#080;font-weight:bold">if</span> options[<span style="color:#A60">:enc</span>].nil?
     puts <span style="color:#036;font-weight:bold">Unisec</span>::<span style="color:#036;font-weight:bold">Hexdump</span>.new(input).display
   <span style="color:#080;font-weight:bold">else</span>
     <span style="color:#777"># using send() is safe here thanks to the value whitelist</span>
-    puts puts <span style="color:#036;font-weight:bold">Unisec</span>::<span style="color:#036;font-weight:bold">Hexdump</span>.send(options[<span style="color:#A60">:enc</span>], input)
+    puts <span style="color:#036;font-weight:bold">Unisec</span>::<span style="color:#036;font-weight:bold">Hexdump</span>.send(options[<span style="color:#A60">:enc</span>], input)
   <span style="color:#080;font-weight:bold">end</span>
 <span style="color:#080;font-weight:bold">end</span></pre>
     </td>
@@ -245,7 +248,7 @@ <h3 class="signature first" id="call-instance_method">
 </div>
 
       <div id="footer">
-  Generated on Tue Jan 23 10:38:47 2024 by
+  Generated on Fri Feb 16 11:14:39 2024 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.3.0).
 </div>