Skip to content

Commit

Permalink
Update Admin API to use the 'jidoteki-admin' repo as a dep for generi…
Browse files Browse the repository at this point in the history
…c functions/json/semver libs
  • Loading branch information
aw committed Mar 7, 2018
1 parent 07f5e7d commit d152079
Show file tree
Hide file tree
Showing 25 changed files with 271 additions and 305 deletions.
8 changes: 5 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ cache: apt
dist: precise

before_script:
- make
- wget http://software-lab.de/picoLisp-16.6.tgz -O /tmp/picolisp.tgz
- cd /tmp; tar -xf /tmp/picolisp.tgz
- cd /tmp/picoLisp/src64 && make
- export PATH=$PATH:/tmp/picoLisp
- git clone https://github.com/on-prem/jidoteki-admin /tmp/jidoteki-admin
- sudo pip install ansible==1.8.4
- cd /tmp/jidoteki-admin && git checkout merge-scripts && make PREFIX_DIR=/tmp

script:
- export PATH=$PATH:/tmp/picoLisp
- cd ${TRAVIS_BUILD_DIR} && make check
- cd ${TRAVIS_BUILD_DIR} && make check PREFIX_DIR=/tmp
150 changes: 150 additions & 0 deletions CHANGELOG-2017.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
# Changelog 2017

## 1.20.0 (2017-12-17)

### New features

* [dashboard] Redesign network settings to select dhcp/static
* [dashboard] Display network 'Mode' in dashboard

### Minor fixes

* [html] Add 'Content-Security-Policy' HTTPS security header
* [html] Remove 'waiting for...' empty messages

### Bug fixes

* [html] NTP server disappears when switching from STATIC to DHCP

## 1.19.0 (2017-11-27)

### Minor fixes

* [html] Append API version to static files
* [api] Ensure custom code is loaded after generic code

## 1.18.3 (2017-11-16)

### Minor fixes

* [html] Ensure HTTPS security headers are sent for downloads and HTML/404 pages
* [dashboard] Set `autocomplete=off` on storage page

## 1.18.2 (2017-11-02)

### Regression fixes

* [dashboard] Alert and status panel displays per section, not globally

## 1.18.1 (2017-11-01)

### Minor fixes

* [dashboard] Ensure `Backup/Restore` section displays the alert and status panel when needed
* [dashboard] Set `autocomplete=off` on login/token page, instead of `autocomplete=false`

## 1.18.0 (2017-10-25)

### Bug fixes

* [api/dashboard] Jidoteki issue #416 - Authenticated API endpoint validation
Certain authenticated API endpoints are not validating the length of strings,
or the type of data which can be submitted. This could lead to unexpected
behaviour or XSS script injection.
All API endpoints and dashboard forms are correctly validated and escaped as of `v1.18.0`

### New features

* [api] Add `/backup` endpoints to create/delete/restore a backup
* [api] Add `/endpoints` endpoint to list all API endpoints
* [api] Add the ability to disable "optional" API endpoints (ex: `/backup`)
* [api] Add audit-logging to authentication and new API calls

### Minor fixes

* [api] Ensure `/health` endpoint is not cached
* [api] Refactor redundant functions
* [api] Add regression tests for new features and validations
* [api] Remove PicoLisp namespaces/symbols in unit tests
* [dashboard] Update Fontello fonts
* [dashboard] Standardize the look and feel of each section
* [dashboard] Add Jidoteki footer to bottom of Dashboard UI
* [dashboard] Split API documentation into two main sections: `default` and `optional` endpoints
* [deps] Update `picolisp-json`, `picolisp-semver`, and `picolisp-unit` dependencies

### Potentially breaking changes

**HTTP security headers are included in every HTTPS request:**

* `Strict-Transport-Security: max-age=31536000 ; includeSubDomains`
* `X-Frame-Options: deny`
* `X-XSS-Protection: 1`
* `X-Content-Type-Options: nosniff`

**All authenticated API endpoints are now validated, and will not accept missing or invalid data:**

* `POST /setup`: `newtoken` parameter only accepts printable ASCII characters (ASCII codes 33-126, no spaces)
This change does not affect existing tokens which may contain non-printable characters
* `POST /settings`: `settings` parameter validates each `network` value:
- **interface (required)**: `a-zA-Z0-9` (alphanumeric), between 3 and 14 characters
(Not required prior to `v1.18.0`)
- **hostname (required)**: `a-zA-Z0-9` (alphanumeric) + `.-`, between 3 and 255 characters
(Not required prior to `v1.18.0`)
- **ip_address**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **netmask**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **gateway**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **dns1**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **dns2**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **ntpserver**: `a-zA-Z0-9` (alphanumeric) + `.-:`, between 3 and 255 characters
* `POST /storage`: `settings` parameter validates each `storage` value:
- [nfs] **mount_options (required)**: `a-zA-Z0-9` (alphanumeric) + `.-=,`, between 3 and 255 characters
- [nfs] **ip (required)**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- [nfs] **share (required)**: `a-zA-Z0-9` (alphanumeric) + `.-_/`, between 3 and 255 characters
- [aoe] **device (required)**: `a-zA-Z0-9` (alphanumeric) + `.-`, between 3 and 255 characters
- [iscsi] **target (required)**: `a-zA-Z0-9` (alphanumeric) + `.-_:`, between 3 and 255 characters
- [iscsi] **ip (required)**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- [iscsi] **username (required)**: printable ASCII characters (ASCII codes 33-126, no spaces), between 3 and 255 characters
- [iscsi] **password (required)**: printable ASCII characters (ASCII codes 33-126, no spaces), between 3 and 255 characters
- [nbd] **export_name (required)**: `a-zA-Z0-9` (alphanumeric) + `.-_/`, between 3 and 255 characters
- [nbd] **ip (required)**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- [nbd] **port (required)**: `0-9` (numeric) + `.-_/`, between 1 and 5 characters

**All "optional" endpoints are disabled by default:**

* Configure the `/usr/local/etc/jidoteki-admin-api.json` file to enable the endpoints
(network settings, certs, license, storage, backup)

**"400 Bad Request" responses are descriptive:**

* Many (not all) `400 Bad Request` responses now contain an `Error-Message` string and HTTP header

## 1.17.0 (2017-10-02)

* Ensure NTP server is configurable and displayed correctly. #20

## 1.16.0 (2017-07-25)

* Ensure '/changelog' returns a no-cache header
* Ensure '/build' returns a no-cache header
* Fix parsing of '/services' with capital T. #30
* Disable all versions of TLS/SSL except TLSv1.2
* Disable form autocomplete on login/token page

## 1.15.0 (2017-04-13)

* Add '/health' endpoint to retrieve information about appliance health
* Add 'picolisp-semver' module dependency
* Display health status at top Admin Dashboard
* Return jqXHR on API call errors for easier debugging
* Fix disk usage graph display. #26

## 1.14.0 (2017-02-10)

* Add '/build' endpoint to retrieve information about the specific build. #28
* Make storage options "optional". #27
* Add extra Storage options, such as NBD, AoE, iSCSI

## 1.13.0 (2017-01-09)

* Add endpoint to upload and update persistent Storage options
* Fix short auth token issue by limiting Token to 1-255 chars #25
148 changes: 6 additions & 142 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,150 +1,14 @@
# Changelog

## 1.20.0 (2017-12-17)
## 1.21.0 (TBD)

### New features

* [dashboard] Redesign network settings to select dhcp/static
* [dashboard] Display network 'Mode' in dashboard
* [api/frontend] Accept IPv6 addresses in Network settings
* [api] Add audit logging to additional API endpoints
* [api] Include update status percentage in "/update" endpoint
* [frontend] Renamed 'Jidoteki' to 'On-Prem'

### Minor fixes

* [html] Add 'Content-Security-Policy' HTTPS security header
* [html] Remove 'waiting for...' empty messages

### Bug fixes

* [html] NTP server disappears when switching from STATIC to DHCP

## 1.19.0 (2017-11-27)

### Minor fixes

* [html] Append API version to static files
* [api] Ensure custom code is loaded after generic code

## 1.18.3 (2017-11-16)

### Minor fixes

* [html] Ensure HTTPS security headers are sent for downloads and HTML/404 pages
* [dashboard] Set `autocomplete=off` on storage page

## 1.18.2 (2017-11-02)

### Regression fixes

* [dashboard] Alert and status panel displays per section, not globally

## 1.18.1 (2017-11-01)

### Minor fixes

* [dashboard] Ensure `Backup/Restore` section displays the alert and status panel when needed
* [dashboard] Set `autocomplete=off` on login/token page, instead of `autocomplete=false`

## 1.18.0 (2017-10-25)

### Bug fixes

* [api/dashboard] Jidoteki issue #416 - Authenticated API endpoint validation
Certain authenticated API endpoints are not validating the length of strings,
or the type of data which can be submitted. This could lead to unexpected
behaviour or XSS script injection.
All API endpoints and dashboard forms are correctly validated and escaped as of `v1.18.0`

### New features

* [api] Add `/backup` endpoints to create/delete/restore a backup
* [api] Add `/endpoints` endpoint to list all API endpoints
* [api] Add the ability to disable "optional" API endpoints (ex: `/backup`)
* [api] Add audit-logging to authentication and new API calls

### Minor fixes

* [api] Ensure `/health` endpoint is not cached
* [api] Refactor redundant functions
* [api] Add regression tests for new features and validations
* [api] Remove PicoLisp namespaces/symbols in unit tests
* [dashboard] Update Fontello fonts
* [dashboard] Standardize the look and feel of each section
* [dashboard] Add Jidoteki footer to bottom of Dashboard UI
* [dashboard] Split API documentation into two main sections: `default` and `optional` endpoints
* [deps] Update `picolisp-json`, `picolisp-semver`, and `picolisp-unit` dependencies

### Potentially breaking changes

**HTTP security headers are included in every HTTPS request:**

* `Strict-Transport-Security: max-age=31536000 ; includeSubDomains`
* `X-Frame-Options: deny`
* `X-XSS-Protection: 1`
* `X-Content-Type-Options: nosniff`

**All authenticated API endpoints are now validated, and will not accept missing or invalid data:**

* `POST /setup`: `newtoken` parameter only accepts printable ASCII characters (ASCII codes 33-126, no spaces)
This change does not affect existing tokens which may contain non-printable characters
* `POST /settings`: `settings` parameter validates each `network` value:
- **interface (required)**: `a-zA-Z0-9` (alphanumeric), between 3 and 14 characters
(Not required prior to `v1.18.0`)
- **hostname (required)**: `a-zA-Z0-9` (alphanumeric) + `.-`, between 3 and 255 characters
(Not required prior to `v1.18.0`)
- **ip_address**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **netmask**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **gateway**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **dns1**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **dns2**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- **ntpserver**: `a-zA-Z0-9` (alphanumeric) + `.-:`, between 3 and 255 characters
* `POST /storage`: `settings` parameter validates each `storage` value:
- [nfs] **mount_options (required)**: `a-zA-Z0-9` (alphanumeric) + `.-=,`, between 3 and 255 characters
- [nfs] **ip (required)**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- [nfs] **share (required)**: `a-zA-Z0-9` (alphanumeric) + `.-_/`, between 3 and 255 characters
- [aoe] **device (required)**: `a-zA-Z0-9` (alphanumeric) + `.-`, between 3 and 255 characters
- [iscsi] **target (required)**: `a-zA-Z0-9` (alphanumeric) + `.-_:`, between 3 and 255 characters
- [iscsi] **ip (required)**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- [iscsi] **username (required)**: printable ASCII characters (ASCII codes 33-126, no spaces), between 3 and 255 characters
- [iscsi] **password (required)**: printable ASCII characters (ASCII codes 33-126, no spaces), between 3 and 255 characters
- [nbd] **export_name (required)**: `a-zA-Z0-9` (alphanumeric) + `.-_/`, between 3 and 255 characters
- [nbd] **ip (required)**: `abcdef0123456789ABCDEF.:`, between 3 and 45 characters
- [nbd] **port (required)**: `0-9` (numeric) + `.-_/`, between 1 and 5 characters

**All "optional" endpoints are disabled by default:**

* Configure the `/usr/local/etc/jidoteki-admin-api.json` file to enable the endpoints
(network settings, certs, license, storage, backup)

**"400 Bad Request" responses are descriptive:**

* Many (not all) `400 Bad Request` responses now contain an `Error-Message` string and HTTP header

## 1.17.0 (2017-10-02)

* Ensure NTP server is configurable and displayed correctly. #20

## 1.16.0 (2017-07-25)

* Ensure '/changelog' returns a no-cache header
* Ensure '/build' returns a no-cache header
* Fix parsing of '/services' with capital T. #30
* Disable all versions of TLS/SSL except TLSv1.2
* Disable form autocomplete on login/token page

## 1.15.0 (2017-04-13)

* Add '/health' endpoint to retrieve information about appliance health
* Add 'picolisp-semver' module dependency
* Display health status at top Admin Dashboard
* Return jqXHR on API call errors for easier debugging
* Fix disk usage graph display. #26

## 1.14.0 (2017-02-10)

* Add '/build' endpoint to retrieve information about the specific build. #28
* Make storage options "optional". #27
* Add extra Storage options, such as NBD, AoE, iSCSI

## 1.13.0 (2017-01-09)

* Add endpoint to upload and update persistent Storage options
* Fix short auth token issue by limiting Token to 1-255 chars #25
* [deps] Remove `json, semver, unit` deps as direct dependencies, add `jidoteki-admin` as dependency
45 changes: 4 additions & 41 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,48 +1,14 @@
# Makefile

PIL_MODULE_DIR ?= .modules
REPO_PREFIX ?= https://github.com/aw

## Edit below
JSON_REPO = $(REPO_PREFIX)/picolisp-json.git
JSON_DIR = $(PIL_MODULE_DIR)/picolisp-json/HEAD
JSON_REF ?= v2.2.0
SEMVER_REPO = $(REPO_PREFIX)/picolisp-semver.git
SEMVER_DIR = $(PIL_MODULE_DIR)/picolisp-semver/HEAD
SEMVER_REF ?= v0.9.0
## Edit above

# Unit testing
TEST_REPO = $(REPO_PREFIX)/picolisp-unit.git
TEST_DIR = $(PIL_MODULE_DIR)/picolisp-unit/HEAD

# Generic
.PHONY: all clean html

all: $(JSON_DIR) $(SEMVER_DIR)

$(JSON_DIR):
mkdir -p $(JSON_DIR) && \
git clone $(JSON_REPO) $(JSON_DIR) && \
cd $(JSON_DIR) && \
git checkout $(JSON_REF) && \
$(MAKE)
.PHONY: all check run-tests html javascript js minify ui

$(SEMVER_DIR):
mkdir -p $(SEMVER_DIR) && \
git clone $(SEMVER_REPO) $(SEMVER_DIR) && \
cd $(SEMVER_DIR) && \
git checkout $(SEMVER_REF) && \
$(MAKE)
all: check

$(TEST_DIR):
mkdir -p $(TEST_DIR) && \
git clone $(TEST_REPO) $(TEST_DIR)

check: all $(TEST_DIR) run-tests
check: run-tests

run-tests:
PIL_NAMESPACES=false ./test.l
JIDO_ADMIN_PATH=$(PREFIX_DIR)/opt/jidoteki/tinyadmin ./test.l

html:
jade -o . -P -E html ui/index.jade
Expand All @@ -57,6 +23,3 @@ minify:
minify docs/ui.js >> docs/ui.min.js

ui: html javascript minify

clean:
rm -rf $(JSON_DIR) $(SEMVER_DIR) $(TEST_DIR)
Loading

0 comments on commit d152079

Please sign in to comment.