Devtoolkit is a powerful and ever-expanding toolkit designed to streamline daily tasks in Golang software development. Within this library, you'll find an array of features, such as tools for working with yml or json prop files, slices, handling generic objects, managing concurrency, and more. As Devtoolkit continues to evolve, it will encompass even more functionalities to cater to a variety of programming needs.
- Devtoolkit
go get github.com/rendis/devtoolkit
ConcurrentExec is a utility for executing a series of functions concurrently.
var fns []devtoolkit.ConcurrentFn
fns = append(fns, func(ctx context.Context) (any, error) {
// Implement function logic
return "Result1", nil
})
fns = append(fns, func(ctx context.Context) (any, error) {
// Implement function logic
return "Result2", nil
})
ctx := context.Background()
ce, err := devtoolkit.NewConcurrentExec().ExecuteFns(ctx, fns...)
if err != nil {
fmt.Println(err)
}
// errors is a slice of errors returned by the functions, where each index corresponds to the function at the same index in the fns slice
errors := ce.Errors()
for _, err := range errors {
if err != nil {
fmt.Println(err)
}
}
// results is a slice of results returned by the functions, where each index corresponds to the function at the same index in the fns slice
// Note: results are of type any, so you'll need to cast them to the appropriate type
results := ce.Results()
for _, res := range results {
fmt.Println(res)
}Note: This example does not include error handling, be sure to do so in your implementations.
ConcurrentWorkers is a utility for executing a series of functions concurrently using a pool of workers.
var maxWorkers = 5
var cw = NewConcurrentWorkers(maxWorkers)
for i := 0; i < 10; i++ {
cw.Execute(func() {
// do something cool
})
}
// Stop the workers with an error
cw.Stop(fmt.Errorf("Something went wrong"))
// Stop the workers without an error
cw.Stop(nil)
// Wait for all workers to finish
cw.Wait()ConcurrentManager is a struct that dynamically manages a pool of workers within set limits.
It adjusts worker count based on load, offering functions to allocate, release, and wait for workers,
optimizing concurrent task handling.
// NewConcurrentManager creates a new instance of ConcurrentManager with specified parameters.
// It ensures that the provided parameters are within acceptable ranges and initializes the manager.
func NewConcurrentManager(minWorkers, maxWorkers int, workerIncreaseRate float64, timeIncreasePeriod time.Duration) *ConcurrentManager
// Allocate requests a new worker to be allocated.
// It blocks if the maximum number of workers has been reached, until a worker is released.
func (c *ConcurrentManager) Allocate()
// Release frees up a worker, making it available for future tasks.
// It only releases a worker if the release condition is met, ensuring resources are managed efficiently.
func (c *ConcurrentManager) Release()
// Wait blocks until all workers have finished their tasks.
// It ensures that all resources are properly cleaned up before shutting down or reinitializing the manager.
func (c *ConcurrentManager) Wait()AtomicNumber is a utility for managing an number atomically.
// initialize an atomic number with default value 0
var atomic AtomicNumber[int]
atomic.Get() // get the current value
atomic.Set(5) // set the value to 5
atomic.Increment() // increment the value by 1
atomic.IncrementBy(10) // add 10 to the value
atomic.IncrementAndGet() // increment the value by 1 and return the new value
atomic.IncrementByAndGet(10) // add 10 to the value and return the new value
var incremented = atomic.IncrementIf(func(v int) {return v > 0}) // increment the value by 1 if the condition is met
var incrementedBy = atomic.IncrementByIf(10, func(v int) {return v > 0}) // add 10 to the value if the condition is met
// same functions are available for decrementing the value
atomic.Decrement() // decrement the value by 1
...Utility functions for loading configuration properties from JSON or YAML files. This functionality supports the injection of environment variables directly into the configuration properties.
LoadPropFile supports field validation using struct tags provided by the go-playground/validator library.
You can register your own custom validators using the RegisterCustomValidator function.
devtoolkit provides the following built-in validators:
trimmed-non-empty- checks whether a string is not empty after trimming whitespace
dbConfig:
host: ${DB_HOST}
port: ${DB_PORT}
username: ${DB_USERNAME}
password: ${DB_PASSWORD}
description: "YAML config file"{
"dbConfig": {
"host": "${DB_HOST}",
"port": "${DB_PORT}",
"username": "${DB_USERNAME}",
"password": "${DB_PASSWORD}",
"description": "JSON config file"
}
}type Config struct {
DBConfig `json:"dbConfig" yaml:"dbConfig" validate:"required"`
}
type DBConfig struct {
Host string `json:"host" yaml:"host" validate:"required"`
Port int `json:"port" yaml:"port" validate:"required,min=1,max=65535"`
Username string `json:"username" yaml:"username" validate:"required,trimmed-non-empty"`
Password string `json:"password" yaml:"password" validate:"required,trimmed-non-empty"`
Description string `json:"description" yaml:"description" validate:"required"`
}
func (p *Config) SetDefaults() {
p.DBConfig.Host = "localhost"
p.DBConfig.Port = 3306
}
func trimmedNonEmpty(fl validator.FieldLevel) bool {
s := fl.Field().String()
trimmed := strings.TrimSpace(s)
return len(trimmed) > 0
}
cfg := &Config{}
props := []devtoolkit.ToolKitProp{cfg}
devtoolkit.RegisterCustomValidator("trimmed-non-empty", trimmedNonEmpty)
err := devtoolkit.LoadPropFile("config.json", props)Utility functions and patterns to ensure resilient operation execution.
The RetryOperation function retries an operation for a specified number of times with optional exponential backoff. It's useful when operations have a tendency to fail temporarily and may succeed on a subsequent attempt.
type ResilienceOptions struct {
MaxRetries int // indicates the maximum number of retries. Default is 3.
WaitTime time.Duration // indicates the wait time between retries. Default is 100ms.
Backoff bool // indicates whether to use exponential backoff. Default is false.
RawError bool // indicates whether to return the raw error or wrap it in a new error. Default is false.
IsIgnorableErrorHandler func(error) bool // indicates whether to ignore the error or not. Default is nil.
ReturnIgnorable bool // indicates whether to return the ignorable error or not. Default is false.
}
func NewResilience(options *ResilienceOptions) (Resilience, error)Example:
operation := func() error {
return networkCall() // Some hypothetical network operation
}
options := &devtoolkit.ResilienceOptions{
MaxRetries: 3,
WaitTime: 2 * time.Second,
Backoff: true,
}
resilienceHandler, err := devtoolkit.NewResilience(options)
if err != nil {
panic(err)
}
err = resilienceHandler.RetryOperation(operation) // wrapped error returned
if err != nil {
fmt.Println("Operation failed.", err)
} else {
fmt.Println("Operation succeeded!")
}With the RetryOperation function, users can easily add resiliency to their operations and ensure that temporary failures don't lead to complete system failures.
ProcessChain is an implementation that enables the orderly execution of operations on data within a Go application.
Leveraging the "Chain of Responsibility" and "Command" design patterns, it allows for the addition of operations (links)
and an optional save step to ensure data integrity after each operation. Ideal for scenarios requiring a series of
actions on data with the flexibility to add, remove, or modify steps as needed.
type ProcessChain[T any] interface {
AddLink(string, LinkFn[T]) error
SetSaveStep(SaveStep[T])
GetChain() []string
Execute(context.Context, T) ([]string, error)
ExecuteWithIgnorableLinks(context.Context, T, []string) ([]string, error)
}
func NewProcessChain[T any](opts *ProcessChainOptions) ProcessChain[T]Example:
type Data struct {
// Your data fields here
}
func step1(d Data) error {
// Define a process operation
}
func step2(d Data) error {
// Define a process operation
}
func saveData(d Data) error {
// Define a save operation
}
chain := NewProcessChain[Data]()
chain.AddLink("step1", step1)
chain.AddLink("step2", step2)
chain.SetSaveStep(saveData)
err := chain.Execute(Data{})The ToPtr function takes a value of any type and returns a pointer to it.
val := 5
ptr := devtoolkit.ToPtr(val)
fmt.Println(*ptr) // Returns 5The IsZero function checks whether a value is the zero value of its type.
fmt.Println(devtoolkit.IsZero(0)) // Returns true
fmt.Println(devtoolkit.IsZero(1)) // Returns false
fmt.Println(devtoolkit.IsZero("")) // Returns trueThe StructToMap function converts a struct to a map[string]any.
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
Email string `json:"email"`
}
p := Person{
Name: "John",
Age: 30,
Email: "john@example.com",
}
personMapData, err := devtoolkit.StructToMap(p)The MapToStruct function converts a map[string]any to a pointer to a struct.
// personMapData is a map[string]any containing the data of a Person struct, see StructToMap example
ptrToNewPerson, err := devtoolkit.MapToStruct[Person](personMapData)CastToPointer casts a value to a pointer of the same type.
func CastToPointer[T any](v any) (*T, bool)Rules:
- v must be a pointer.
- if v not a pointer, returns false.
- if v is nil, returns false.
- if v is a pointer but not of the given type, returns false.
- if v is a pointer of the given type, returns true.
IfThenElse returns the first value if the condition is true, otherwise it returns the second value.
func IfThenElse[T any](condition bool, first, second T) TIfThenElseFn returns the first value if the condition is true, otherwise it returns the result of the second value.
func IfThenElseFn[T any](condition bool, first T, second func() T) TDefaultIfNil returns the first value if it is not nil, otherwise it returns the second value.
func DefaultIfNil[T any](first *T, second T) TZeroValue returns the zero value of a type.
func ZeroValue[T any]() TToInt converts a value to an int.
func ToInt(value any) (int, bool)ToFloat64 converts a value to a float64.
func ToFloat64(value any) (float64, bool)StrToStruct converts a string to a struct.
func StrToStruct[T any](s string) (*T, error)The Pair type represents a pair of values.
type Pair[F any, S any] struct {
First F
Second S
}The Triple type represents a triple of values.
type Triple[F any, S any, T any] struct {
First F
Second S
Third T
}The CSV reader provides a simple and efficient way to read CSV files in Go.
More details can be found in the CSV Reader documentation.
The struct-guard tool generates wrapper structs in Go for tracking changes to the fields of the original struct.
More details can be found in the struct-generator documentation.
Common utility functions for working with slices.
The Contains function checks whether a slice contains an item. The item must be comparable.
func Contains[T comparable](slice []T, item T) boolExample:
slice := []int{1, 2, 3}
item := 2
fmt.Println(devtoolkit.Contains(slice, item)) // Returns trueThe ContainsWithPredicate function checks whether a slice contains an item using a predicate to compare items.
func ContainsWithPredicate[T any](slice []T, item T, predicate func(T, T) bool) boolExample:
slice := []int{1, 2, 3}
item := 2
predicate := func(a, b int) bool { return a == b }
fmt.Println(devtoolkit.ContainsWithPredicate(slice, item, predicate)) // Returns trueIndexOf returns the index of the first instance of an item in a slice, or -1 if the item is not present in the slice.
func IndexOf[T comparable](slice []T, item T) intExample:
index := IndexOf([]int{1, 2, 3, 2, 1}, 2)
fmt.Println(index) // Output: 1IndexOfWithPredicate returns the index of the first instance of an item in a slice, or -1 if the item is not present in the slice. It uses a predicate function to compare items.
func IndexOfWithPredicate[T any](slice []T, item T, predicate func(T, T) bool) intExample:
index := IndexOfWithPredicate([]string{"apple", "banana", "cherry"}, "APPLE", strings.EqualFold)
fmt.Println(index) // Output: 0LastIndexOf returns the index of the last instance of an item in a slice, or -1 if the item is not present in the slice.
func LastIndexOf[T comparable](slice []T, item T) intExample:
index := LastIndexOf([]int{1, 2, 3, 2, 1}, 2)
fmt.Println(index) // Output: 3LastIndexOfWithPredicate returns the index of the last instance of an item in a slice, or -1 if the item is not present in the slice. It uses a predicate function to compare items.
func LastIndexOfWithPredicate[T any](slice []T, item T, predicate func(T, T) bool) intExample:
index := LastIndexOfWithPredicate([]string{"apple", "banana", "cherry", "apple"}, "APPLE", strings.EqualFold)
fmt.Println(index) // Output: 3Remove removes the first instance of an item from a slice, if present. It returns true if the item was removed, false otherwise.
func Remove[T comparable](slice []T, item T) boolExample:
removed := Remove([]int{1, 2, 3, 2, 1}, 2)
fmt.Println(removed) // Output: trueRemoveWithPredicate removes the first instance of an item from a slice, if present. It uses a predicate function to compare items. It returns true if the item was removed, false otherwise.
func RemoveWithPredicate[T any](slice []T, item T, predicate func(T, T) bool) boolExample:
removed := RemoveWithPredicate([]string{"apple", "banana", "cherry"}, "APPLE", strings.EqualFold)
fmt.Println(removed) // Output: trueRemoveAll removes all instances of an item from a slice, if present. It returns true if the item was removed, false otherwise.
func RemoveAll[T comparable](slice []T, item T) boolExample:
removed := RemoveAll([]int{1, 2, 3, 2, 1}, 2)
fmt.Println(removed) // Output: trueRemoveAllWithPredicate removes all instances of an item from a slice, if present. It uses a predicate function to compare items. It returns true if the item was removed, false otherwise.
func RemoveAllWithPredicate[T any](slice []T, item T, predicate func(T, T) bool) boolExample:
removed := RemoveAllWithPredicate([]string{"apple", "banana", "cherry", "apple"}, "APPLE", strings.EqualFold)
fmt.Println(removed) // Output: trueRemoveAt removes the item at a given index from a slice. It returns true if the item was removed, false otherwise.
func RemoveAt[T any](slice []T, index int) boolExample:
removed := RemoveAt([]int{1, 2, 3}, 1)
fmt.Println(removed) // Output: trueRemoveRange removes the items in a given range from a slice. It returns true if items were removed, false otherwise.
func RemoveRange[T any](slice []T, start, end int) boolExample:
removed := RemoveRange([]int{1, 2, 3, 4, 5}, 1, 3)
fmt.Println(removed) // Output: trueRemoveIf removes all items from a slice for which a predicate function returns true. It returns true if any items were removed, false otherwise.
func RemoveIf[T any](slice []T, predicate func(T) bool) boolExample:
removed := RemoveIf([]int{1, 2, 3, 4, 5}, func(n int) bool { return n%2 == 0 })
fmt.Println(removed) // Output: trueFilter returns a new slice containing all items from the original slice for which a predicate function returns true.
func Filter[T any](slice []T, predicate func(T) bool) []TExample:
filtered := Filter([]int{1, 2, 3, 4, 5}, func(n int) bool { return n%2 == 0 })
fmt.Println(filtered) // Output: [2 4]FilterNot returns a new slice containing all items from the original slice for which a predicate function returns false.
func FilterNot[T any](slice []T, predicate func(T) bool) []TExample:
filtered := FilterNot([]int{1, 2, 3, 4, 5}, func(n int) bool { return n%2 == 0 })
fmt.Println(filtered) // Output: [1 3 5]Map applies a transformation function to all items in a slice and returns a new slice containing the results.
func Map[T, R any](slice []T, mapper func(T) R) []R Example:
mapped := Map([]int{1, 2, 3}, func(n int) int { return n * 2 })
fmt.Println(mapped) // Output: [2 4 6]RemoveDuplicates removes all duplicate items from a slice. It returns true if any items were removed, false otherwise.
func RemoveDuplicates[T comparable](slice []T) boolExample:
removed := RemoveDuplicates([]int{1, 2, 3, 2, 1})
fmt.Println(removed) // Output: trueReverse reverses the order of items in a slice.
func Reverse[T any](slice []T)Example:
data := []int{1, 2, 3}
Reverse(data)
fmt.Println(data) // Output: [3 2 1]Difference returns a new slice containing all items from the original slice that are not present in the other slice.
func Difference[T comparable](slice, other []T) []TExample:
diff := Difference([]int{1, 2, 3, 4, 5}, []int{3, 4, 5, 6, 7})
fmt.Println(diff) // Output: [1 2]Intersection returns a new slice containing all items from the original slice that are also present in the other slice.
func Intersection[T comparable](slice, other []T) []TExample:
inter := Intersection([]int{1, 2, 3, 4, 5}, []int{3, 4, 5, 6, 7})
fmt.Println(inter) // Output: [3 4 5]Union returns a new slice containing all unique items from both the original slice and the other slice.
func Union[T comparable](slice, other []T) []TExample:
union := Union([]int{1, 2, 3}, []int{3, 4, 5})
fmt.Println(union) // Output: [1 2 3 4 5]GetMapKeys returns a slice of keys from a map.
func GetMapKeys[K comparable, V any](m map[K]V) []KExample:
keys := GetMapKeys(map[string]int{"a": 1, "b": 2})
fmt.Println(keys) // Output: [a b]GetMapValues returns a slice of values from a map.
func GetMapValues[K comparable, V any](m map[K]V, removeDuplicates bool) []VExample:
values := GetMapValues(map[string]int{"a": 1, "b": 2, "c": 1}, false)
fmt.Println(values) // Output: [1 2 1]
values = GetMapValues(map[string]int{"a": 1, "b": 2, "c": 1}, true)
fmt.Println(values) // Output: [1 2]Contributions to this library are welcome. Please open an issue to discuss the enhancement or feature you would like to add, or just make a pull request.
Devtoolkit is licensed under the MIT License. Please see the LICENSE file for details.