add confusables

Acceis · Jul 21, 2023 · 573dc38 · 573dc38
1 parent 634f539
commit 573dc38
Show file tree

Hide file tree

Showing 47 changed files with 1,654 additions and 61 deletions.
diff --git a/Gemfile b/Gemfile
@@ -15,6 +15,7 @@ end
 group :runtime, :all do
   gem 'ctf-party', '~> 2.3' # string conversion
   gem 'twitter_cldr', '~> 6.11', '>= 6.11.5' # ICU / CLDR
+  gem 'unicode-confusable', '~> 1.9' # confusable chars
 end
 
 # Needed to install dependencies

diff --git a/Gemfile.lock b/Gemfile.lock
@@ -6,6 +6,7 @@ PATH
       dry-cli (~> 1.0)
       paint (~> 2.3)
       twitter_cldr (~> 6.11, >= 6.11.5)
+      unicode-confusable (~> 1.9)
 
 GEM
   remote: https://rubygems.org/
@@ -55,6 +56,7 @@ GEM
       tzinfo
     tzinfo (2.0.6)
       concurrent-ruby (~> 1.0)
+    unicode-confusable (1.9.0)
     unicode-display_width (2.4.2)
     webrick (1.8.1)
     yard (0.9.34)
@@ -76,6 +78,7 @@ DEPENDENCIES
   rake (~> 13.0)
   rubocop (~> 1.53)
   twitter_cldr (~> 6.11, >= 6.11.5)
+  unicode-confusable (~> 1.9)
   unisec!
   webrick (~> 1.8, >= 1.8.1)
   yard (>= 0.9.27, < 0.10)

diff --git a/README.md b/README.md
@@ -18,7 +18,17 @@ A CLI tool and library to play with Unicode security.
 
 ## Features
 
-- Code point ↔️ Surrogates
+- **Confusables / homoglyphs**
+  - List confusables characters for a given character
+  - Replace all characters from a string with random confusables
+- **Hexdump**
+  - UTF-8, UTF-16, UTF-32 hexadecimal dumps
+- **Properties**
+  - Get all properties of a given Unicode character
+  - List code points matching a Unicode property
+  - List all Unicode properties name
+- **Surrogates**
+  - Code point ↔️ Surrogates conversion
 
 ## Installation
 

diff --git a/docs/pages/usage.md b/docs/pages/usage.md
@@ -2,11 +2,16 @@
 
 ## CLI
 
+### General help
+
 List commands:
 
 ```
 $ unisec --help
 Commands:
+  unisec confusables [SUBCOMMAND]
+  unisec hexdump INPUT                            # Hexdump in all Unicode encodings
+  unisec properties [SUBCOMMAND]
   unisec surrogates [SUBCOMMAND]
 ```
 
@@ -40,8 +45,27 @@ Options:
   --help, -h                        # Print this help
 ```
 
-Also find CLI examples in [the library documentation](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands), e.g. [surrogates](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Surrogates/From).
+### Examples
+
+- **Confusables**
+  - [List](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Confusables/List)
+  - [Randomize](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Confusables/Randomize)
+- [Hexdump](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Hexdump)
+- **Properties**
+  - [Char](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Properties/Char)
+  - [Codepoints](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Properties/Codepoints)
+  - [List](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Properties/List)
+- **Surrogates**
+  - [From](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Surrogates/From)
+  - [To](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands/Surrogates/To)
+
+[Library documentation for commands](https://acceis.github.io/unisec/yard/Unisec/CLI/Commands).
 
 ## Library
 
-See examples in [the library documentation](https://acceis.github.io/unisec/yard/Unisec), e.g. [surrogates](https://acceis.github.io/unisec/yard/Unisec/Surrogates).
+See examples in [the library documentation](https://acceis.github.io/unisec/yard/Unisec).
+
+- [Unisec::Confusables](https://acceis.github.io/unisec/yard/Unisec/Confusables)
+- [Unisec::Hexdump](https://acceis.github.io/unisec/yard/Unisec/Hexdump)
+- [Unisec::Properties](https://acceis.github.io/unisec/yard/Unisec/Properties)
+- [Unisec::Surrogates](https://acceis.github.io/unisec/yard/Unisec/Surrogates)
diff --git a/docs/yard/Integer.html b/docs/yard/Integer.html
@@ -304,7 +304,7 @@ <h3 class="signature " id="to_hex-instance_method">
 </div>
 
       <div id="footer">
-  Generated on Thu Jul 20 16:52:23 2023 by
+  Generated on Fri Jul 21 12:37:32 2023 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.2.0).
 </div>

diff --git a/docs/yard/Unisec.html b/docs/yard/Unisec.html
@@ -79,7 +79,7 @@
   <dl>
     <dt>Defined in:</dt>
     <dd>lib/unisec/utils.rb<span class="defines">,<br />
-  lib/unisec/cli/cli.rb,<br /> lib/unisec/hexdump.rb,<br /> lib/unisec/version.rb,<br /> lib/unisec/properties.rb,<br /> lib/unisec/surrogates.rb,<br /> lib/unisec/cli/hexdump.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb</span>
+  lib/unisec/cli/cli.rb,<br /> lib/unisec/hexdump.rb,<br /> lib/unisec/version.rb,<br /> lib/unisec/properties.rb,<br /> lib/unisec/surrogates.rb,<br /> lib/unisec/cli/hexdump.rb,<br /> lib/unisec/confusables.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb,<br /> lib/unisec/cli/confusables.rb</span>
 </dd>
   </dl>
 
@@ -93,7 +93,7 @@ <h2>Defined Under Namespace</h2>
 
 
 
-      <strong class="classes">Classes:</strong> <span class='object_link'><a href="Unisec/Hexdump.html" title="Unisec::Hexdump (class)">Hexdump</a></span>, <span class='object_link'><a href="Unisec/Properties.html" title="Unisec::Properties (class)">Properties</a></span>, <span class='object_link'><a href="Unisec/Surrogates.html" title="Unisec::Surrogates (class)">Surrogates</a></span>
+      <strong class="classes">Classes:</strong> <span class='object_link'><a href="Unisec/Confusables.html" title="Unisec::Confusables (class)">Confusables</a></span>, <span class='object_link'><a href="Unisec/Hexdump.html" title="Unisec::Hexdump (class)">Hexdump</a></span>, <span class='object_link'><a href="Unisec/Properties.html" title="Unisec::Properties (class)">Properties</a></span>, <span class='object_link'><a href="Unisec/Surrogates.html" title="Unisec::Surrogates (class)">Surrogates</a></span>
 
 
 </p>
@@ -135,7 +135,7 @@ <h2>
 </div>
 
       <div id="footer">
-  Generated on Thu Jul 20 16:52:23 2023 by
+  Generated on Fri Jul 21 12:37:32 2023 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.2.0).
 </div>

diff --git a/docs/yard/Unisec/CLI.html b/docs/yard/Unisec/CLI.html
@@ -79,7 +79,7 @@
   <dl>
     <dt>Defined in:</dt>
     <dd>lib/unisec/cli/cli.rb<span class="defines">,<br />
-  lib/unisec/cli/hexdump.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb</span>
+  lib/unisec/cli/hexdump.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb,<br /> lib/unisec/cli/confusables.rb</span>
 </dd>
   </dl>
 
@@ -117,7 +117,7 @@ <h2>Overview</h2><div class="docstring">
 </div>
 
       <div id="footer">
-  Generated on Thu Jul 20 16:52:23 2023 by
+  Generated on Fri Jul 21 12:37:32 2023 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.2.0).
 </div>

diff --git a/docs/yard/Unisec/CLI/Commands.html b/docs/yard/Unisec/CLI/Commands.html
@@ -84,7 +84,7 @@
   <dl>
     <dt>Defined in:</dt>
     <dd>lib/unisec/cli/cli.rb<span class="defines">,<br />
-  lib/unisec/cli/hexdump.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb</span>
+  lib/unisec/cli/hexdump.rb,<br /> lib/unisec/cli/properties.rb,<br /> lib/unisec/cli/surrogates.rb,<br /> lib/unisec/cli/confusables.rb</span>
 </dd>
   </dl>
 
@@ -104,7 +104,7 @@ <h2>Overview</h2><div class="docstring">
 <p class="children">
 
 
-      <strong class="modules">Modules:</strong> <span class='object_link'><a href="Commands/Properties.html" title="Unisec::CLI::Commands::Properties (module)">Properties</a></span>, <span class='object_link'><a href="Commands/Surrogates.html" title="Unisec::CLI::Commands::Surrogates (module)">Surrogates</a></span>
+      <strong class="modules">Modules:</strong> <span class='object_link'><a href="Commands/Confusables.html" title="Unisec::CLI::Commands::Confusables (module)">Confusables</a></span>, <span class='object_link'><a href="Commands/Properties.html" title="Unisec::CLI::Commands::Properties (module)">Properties</a></span>, <span class='object_link'><a href="Commands/Surrogates.html" title="Unisec::CLI::Commands::Surrogates (module)">Surrogates</a></span>
 
 
 
@@ -125,7 +125,7 @@ <h2>Overview</h2><div class="docstring">
 </div>
 
       <div id="footer">
-  Generated on Thu Jul 20 16:52:23 2023 by
+  Generated on Fri Jul 21 12:37:32 2023 by
   <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
   0.9.34 (ruby-3.2.0).
 </div>

diff --git a/docs/yard/Unisec/CLI/Commands/Confusables.html b/docs/yard/Unisec/CLI/Commands/Confusables.html
@@ -0,0 +1,125 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1.0">
+<title>
+  Module: Unisec::CLI::Commands::Confusables
+
+    &mdash; Documentation by YARD 0.9.34
+
+</title>
+
+  <link rel="stylesheet" href="../../../css/style.css" type="text/css" />
+
+  <link rel="stylesheet" href="../../../css/common.css" type="text/css" />
+
+<script type="text/javascript">
+  pathId = "Unisec::CLI::Commands::Confusables";
+  relpath = '../../../';
+</script>
+
+
+  <script type="text/javascript" charset="utf-8" src="../../../js/jquery.js"></script>
+
+  <script type="text/javascript" charset="utf-8" src="../../../js/app.js"></script>
+
+
+  </head>
+  <body>
+    <div class="nav_wrap">
+      <iframe id="nav" src="../../../class_list.html?1"></iframe>
+      <div id="resizer"></div>
+    </div>
+
+    <div id="main" tabindex="-1">
+      <div id="header">
+        <div id="menu">
+
+    <a href="../../../_index.html">Index (C)</a> &raquo;
+    <span class='title'><span class='object_link'><a href="../../../Unisec.html" title="Unisec (module)">Unisec</a></span></span> &raquo; <span class='title'><span class='object_link'><a href="../../CLI.html" title="Unisec::CLI (module)">CLI</a></span></span> &raquo; <span class='title'><span class='object_link'><a href="../Commands.html" title="Unisec::CLI::Commands (module)">Commands</a></span></span>
+     &raquo; 
+    <span class="title">Confusables</span>
+
+</div>
+
+        <div id="search">
+
+    <a class="full_list_link" id="class_list_link"
+        href="../../../class_list.html">
+
+        <svg width="24" height="24">
+          <rect x="0" y="4" width="24" height="4" rx="1" ry="1"></rect>
+          <rect x="0" y="12" width="24" height="4" rx="1" ry="1"></rect>
+          <rect x="0" y="20" width="24" height="4" rx="1" ry="1"></rect>
+        </svg>
+    </a>
+
+</div>
+        <div class="clear"></div>
+      </div>
+
+      <div id="content"><h1>Module: Unisec::CLI::Commands::Confusables
+
+
+
+</h1>
+<div class="box_info">
+
+
+
+
+
+
+
+
+
+
+
+  <dl>
+    <dt>Defined in:</dt>
+    <dd>lib/unisec/cli/confusables.rb</dd>
+  </dl>
+
+</div>
+
+<h2>Overview</h2><div class="docstring">
+  <div class="discussion">
+    <p>CLI sub-commands <code>unisec confusables xxx</code> for the class <span class='object_link'><a href="../../Confusables.html" title="Unisec::Confusables (class)">Unisec::Confusables</a></span> from the lib.</p>
+
+
+  </div>
+</div>
+<div class="tags">
+
+
+</div><h2>Defined Under Namespace</h2>
+<p class="children">
+
+
+
+
+      <strong class="classes">Classes:</strong> <span class='object_link'><a href="Confusables/List.html" title="Unisec::CLI::Commands::Confusables::List (class)">List</a></span>, <span class='object_link'><a href="Confusables/Randomize.html" title="Unisec::CLI::Commands::Confusables::Randomize (class)">Randomize</a></span>
+
+
+</p>
+
+
+
+
+
+
+
+
+
+</div>
+
+      <div id="footer">
+  Generated on Fri Jul 21 12:37:32 2023 by
+  <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
+  0.9.34 (ruby-3.2.0).
+</div>
+
+    </div>
+  </body>
+</html>