Every wondered what open source resources you should know about when starting your own IT driven post-production/vfx studio? This list contains important tools, projects and links to create, setup, monitor and enhance your infrastructure. Inspired by Awesome-sysadmin.
- studio-sauce
- Backups
- Cloning
- Cloud Computing
- Cloud Storage
- Code Review
- Collaborative Software
- Configuration Management Database
- Configuration Management
- Continuous Integration & Continuous Deployment
- Distributed Filesystems
- DNS
- Hosting Control Panels
- IMAP/POP3
- IT Asset Management
- LDAP
- Log Management
- Monitoring
- Metric & Metric Collection
- Network Configuration Management
- Newsletter
- NoSQL
- Packaging
- Queuing
- RDBMS
- Security
- Service Discovery
- SMTP
- Software Containers
- SSH
- Statistics
- Ticketing systems
- Troubleshooting
- Project Management
- Version control
- Virtualization
- VPN
- XMPP
- Web
- Webmails
- Wikis
- Resources
- Contributing
Backup software.
- Amanda - Client-server model backup tool.
- Bacula - Another Client-server model backup tool.
- Backupninja - Lightweight, extensible meta-backup system.
- Backuppc - Client-server model backup tool with file pooling scheme.
- Burp - Network backup and restore program.
- Duplicity - Encrypted bandwidth-efficient backup using the rsync algorithm.
- Lsyncd - Watches a local directory trees for changes, and then spawns a process to synchronize the changes. Uses rsync by default.
- Rsnapshot - Filesystem Snapshotting Utility.
- SafeKeep - Centralized pull-based backup using
rdiff-backup. - TarSnap - Secure backup service with an open-source client.
- UrBackup - Another client-server backup system.
- DREBS - AWS EBS backup script that supports strategies.
Cloning software.
- Clonezilla - Partition and disk imaging/cloning program.
- Fog - Another computer cloning solution.
- Redo Backup - Easy Backup, Recovery and Restore.
- AppScale - Open source cloud software with Google App Engine compatibility.
- Archipel - Manage and supervise virtual machines using Libvirt.
- CloudStack - Cloud computing software for creating, managing, and deploying infrastructure cloud services.
- Cobbler - Cobbler is a Linux installation server that allows for rapid setup of network installation environments.
- Eucalyptus - Open source private cloud software with AWS compatibility.
- Mesos - Develop and run resource-efficient distributed systems.
- OpenNebula - An user-driven cloud management platform for sysadmins and devops.
- OpenStack - Open source software for building private and public clouds.
- The Foreman - Foreman is a complete lifecycle management tool for physical and virtual servers. FOSS.
- BOSH - IaaS orchestration platform originally written for deploying and managing Cloud Foundry PaaS, but also useful for general purpose distributed systems.
- Cloudify - Open source TOSCA-based cloud orchestration software platform written in Python and YAML.
- Juju - Cloud orechestration tool which manages services as charms, YAML configuration and deployment script bundles.
- MCollective - Ruby framework to manage server orchestration, developed by Puppet labs.
- Overcast - Deploy VMs across different cloud providers, and run commands and scripts across any or all of them in parallel via SSH.
- Rundeck - Simple orchestration tool.
- Salt - It's written in Python.
- git-annex assistant - A synchronised folder on each of your OSX and Linux computers, Android devices, removable drives, NAS appliances, and cloud services.
- ownCloud - Provides universal access to your files via the web, your computer or your mobile devices.
- Seafile - Another Open Source Cloud Storage solution.
- SparkleShare - Provides cloud storage and file synchronization services. By default, it uses Git as a storage backend.
- Swift - A highly available, distributed, eventually consistent object/blob store.
- Syncthing - Open Source system for private, encrypted and authenticated distrobution of data.
Web Based collaborative code review system.
- Gerrit - Based on the Git version control, it facilitates software developers to review modifications to the source code and approve or reject those changes.
- Review Board - Available as free software uner the MIT License.
Collaborative software or groupware suites.
- Citadel/UX - Collaboration suite (messaging and groupware) that is descended from the Citadel family of programs.
- EGroupware - Groupware software written in PHP.
- Horde Groupware - PHP based collaborative software suite that includes email, calendars, wikis, time tracking and file management.
- Kolab - Another groupware suite.
- SOGo - Collaborative software server with a focus on simplicity and scalability.
- Zimbra - Collaborative software suite, that includes an email server and web client.
Configuration management database (CMDB) software.
- i-doit - Open Source IT Documentation and CMDB.
- iTop - A complete open source, ITIL, web based service management tool.
- Ralph - Asset management, DCIM and CMDB system for large Data Centers as well as smaller LAN networks.
- Clusto - Helps you keep track of your inventory, where it is, how it's connected, and provides an abstracted interface for interacting with the elements of the infrastructure.
Configuration management tools.
- Ansible - It's written in Python and manages the nodes over SSH.
- CFEngine - Lightweight agent system. Configuration state is specified via a declarative language.
- Chef - It's written in Ruby and Erlang and uses a pure-Ruby DSL.
- Fabric - Python library and cli tool for streamlining the use of SSH for application deployment or systems administration tasks.
- Pallet - Infrastructure definition, configuration and management via a Clojure DSL.
- Puppet - It's written in Ruby and uses Puppet's declarative language or a Ruby DSL.
- Salt - It's written in Python.
- Slaughter - It's written in Perl.
Continuous integration/deployment software.
- Buildbot - Python-based toolkit for continuous integration.
- Drone - Continuous integration server built on Docker and configured using YAML files.
- GitLab CI - Based off of ruby. They also provide GitLab, which manages git repositories.
- Go - Open source continuous delivery server.
- Jenkins - An extendable open source continuous integration server.
- Vlad the Deployer - Deployment automation.
Network distributed filesystems.
- Ceph - Distributed object store and file system.
- DRBD - Disributed Replicated Block Device.
- LeoFS - Unstructured object/data storage and a highly available, distributed, eventually consistent storage system.
- GlusterFS - Scale-out network-attached storage file system.
- HDFS - Distributed, scalable, and portable file-system written in Java for the Hadoop framework.
- Lustre - A type of parallel distributed file system, generally used for large-scale cluster computing.
- MooseFS - Fault tolerant, network distributed file system.
- MogileFS - Application level, network distributed file system.
- OpenAFS - Distributed network file system with read-only replicas and multi-OS support.
- TahoeLAFS - secure, decentralized, fault-tolerant, peer-to-peer distributed data store and distributed file system.
- XtreemFS - XtreemFS is a fault-tolerant distributed file system for all storage needs.
DNS servers.
- Bind - The most widely used name server software.
- djbdns - A collection of DNS applications, including tinydns.
- Designate - DNS REST API that support several DNS servers as its backend.
- dnsmasq - A lightweight service providing DNS, DHCP and TFTP services to small-scale networks.
- Knot - High performance authoritative-only DNS server.
- NSD - Authoritative only, high performance, simple name server.
- PowerDNS - DNS server with a variety of data storage back-ends and load balancing features.
- Unbound - Validating, recursive, and caching DNS resolver.
- Yadifa - Lightweight authoritative Name Server with DNSSEC capabilities powering the .eu top-level domain.
Web hosting control panels
- Ajenti - Control panel for Linux and BSD.
- Feathur - VPS Provisioning and Management Software.
- ISPConfig - Hosting control panel for Linux.
- VestaCP - Hosting panel for Linux but with Nginx.
- Virtualmin - Control panel for Linux based on webmin.
- ZPanel - Control panel for Linux, BSD, and Windows.
IMAP/POP3 mail servers.
- Courier IMAP/POP3 - Fast, scalable, enterprise IMAP and POP3 server.
- Cyrus IMAP/POP3 - Intended to be run on sealed servers, where normal users are not permitted to log in.
- Dovecot - IMAP and POP3 server written primarily with security in mind.
- Qpopper - One of the oldest and most popular server implementations of POP3.
IT Assets Management software.
- GLPI - Information Resource-Manager with an additional Administration Interface.
- OCS Inventory NG - Enables users to inventory their IT assets.
- RackTables - Datacenter and server room asset management like document hardware assets, network addresses, space in racks, networks configuration.
- Ralph - Asset management, DCIM and CMDB system for large Data Centers as well as smaller LAN networks.
- Snipe IT - Asset & license management software.
LDAP servers.
- 389 Directory Server - Developed by Red Hat.
- Apache Directory Server - Apache Software Foundation project written in Java.
- Fusion Directory - Improve the Management of the services and the company directory based on OpenLDAP.
- OpenDJ - Fork of OpenDS.
- OpenDS - Another directory server written in Java.
- OpenLDAP - Developed by the OpenLDAP Project.
Log management tools: collect, parse, visualize ...
- Elasticsearch - A Lucene Based Document store mainly used for log indexing, storage and analysis.
- Fluentd - Log Collector and Shipper.
- Flume - Distributed log collection and aggregation system.
- Graylog2 - Pluggable Log and Event Analysis Server with Alerting options.
- Heka - Stream processing system which may be used for log aggregation.
- Kibana - Visualize logs and time-stamped data.
- Logstash - Tool for managing events and logs.
- Octopussy - Log Management Solution (Visualize / Alert / Report).
Monitoring software.
- Cacti - Web-based network monitoring and graphing tool.
- Cabot - Monitoring and alerts, similar to PagerDuty.
- check_mk - Collection of extensions for Nagios.
- Dash - A low-overhead monitoring web dashboard for a GNU/Linux machine.
- Icinga - Fork of Nagios.
- LibreNMS - fork of Observium.
- Monit - Small Open Source utility for managing and monitoring Unix systems.
- Munin - Networked resource monitoring tool.
- Naemon - Network monitoring tool based on the Nagios 4 core with performance enhancements and new features.
- Nagios - Computer system, network and infrastructure monitoring software application.
- Observium - SNMP monitoring for servers and networking devices. Runs on linux.
- OMD - The Open Monitoring Distribution.
- Opsview - Based on Nagios 4, Opsview Core is ideal for small IT and test environments.
- Riemann - Flexible and fast events processor allowing complex events/metrics analysis.
- Sensu - Open source monitoring framework.
- Sentry - Application monitoring, event logging and aggregation.
- Shinken - Another monitoring framework.
- Thruk - Multibackend monitoring webinterface with support for Naemon, Nagios, Icinga and Shinken.
- Xymon - Network monitoring inspired by Big Brother.
- Zabbix - Enterprise-class software for monitoring of networks and applications.
- Zenoss - Application, server, and network management platform based on Zope.
Metric gathering and display software.
- Collectd - System statistic collection daemon.
- Collectl - High precision system performance metrics collecting tool.
- Dashing - Ruby gem that allows for rapid statistical dashboard development. An all HTML5 approach allows for big screen displays in data centers or conference rooms.
- Diamond - Python based statistic collection daemon.
- Ganglia - High performance, scalable RRD based monitoring for grids and/or clusters of servers. Compatible with Graphite using a single collection process.
- Grafana - A Graphite & InfluxDB Dashboard and Graph Editor.
- Graphite - Open source scaleable graphing server.
- InfluxDB - Open source distributed time series database with no external dependencies.
- KairosDB - Fast distributed scalable time series database, fork of OpenTSDB 1.x.
- OpenTSDB - Store and server massive amounts of time series data without losing granularity.
- RRDtool - Open source industry standard, high performance data logging and graphing system for time series data.
- Statsd - Application statistic listener.
Network configuration management tools.
- GestióIP - An automated web based IPv4/IPv6 IP Address Management tool.
- RANCID - Monitors network device's configurarion and maintain history of changes.
- rConfig - Another network device configuration management tool.
Newsletter software.
NoSQL databases.
- Column-Family
- Apache HBase - Hadoop database, a distributed, big data store.
- Cassandra - Distributed DBMS designed to handle large amounts of data across many servers.
- Hypertable - C++ based BigTable-like DBMS, communicates through Thrift and runs either as stand-alone or on distributed FS such as Hadoop.
- Document Store
- CouchDB - Ease of use, with multi-master replication document-oriented database system.
- ElasticSearch - Java based database, popular with log aggregation, and email archiving projects.
- MongoDB - Another document-oriented database system.
- RavenDB - Document based database with ACID/Transactional features.
- RethinkDB - Open source distributed document store database, focuses on JSON.
- Graph
- Key-Value
Comparison of NoSQL servers: http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
- fpm - Versatile multi format package creator.
- omnibus-ruby - Full stack, cross distro packaging software (Ruby).
- packman - Full stack, cross distro packaging software (Python).
- tito - Builds RPMs for git-based projects.
- BeanstalkD - A simple, fast work queue.
- Gearman - Fast multi-language queuing/job processing platform.
- NSQ - A realtime distributed messaging platform.
- RabbitMQ - Robust, fully featured, cross distro queuing system.
- ZeroMQ - Lightweight queuing system.
Relational DBMS.
- Firebird - True universal open source database.
- Galera - Galera Cluster for MySQL is an easy-to-use high-availability solution with high system up-time, no data loss, and scalability for future growth.
- MariaDB - Community-developed fork of the MySQL.
- MySQL - Most popular RDBMS server.
- Percona Server - Enhanced, drop-in MySQL replacement.
- PostgreSQL - Object-relational database management system (ORDBMS).
- PostgreSQL-XL - Scalable Open Source PostgreSQL-based database cluster.
- SQLite - Library that implements a self-contained, serverless, zero-configuration, transactional SQL DBS.
Security tools.
- Denyhosts - Thwart SSH dictionary based attacks and brute force attacks.
- Fail2Ban - Scans log files and takes action on IPs that show malicious behavior.
- SpamAssassin - A powerful and popular email spam filter employing a variety of detection techniques.
- Consul - Consul is a tool for service discovery, monitoring and configuration.
- Doozerd - Doozer is a highly-available, completely consistent store for small amounts of extremely important data.
- ZooKeeper - ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.
SMTP servers.
- Exim - Message transfer agent (MTA) developed at the University of Cambridge.
- Haraka - A high-performance, pluginable SMTP server written in JavaScript.
- MailCatcher - Ruby gem that deploys a simply SMTP MTA gateway that accepts all mail and displays in web interface. Useful for debugging or development.
- Maildrop - Open Source disposable email SMTP server, also useful for development.
- OpenSMTPD - Secure SMTP server implementation from the OpenBSD project.
- Postfix - Fast, easy to administer, and secure Sendmail replacement.
- Qmail - Secure Sendmail replacement.
- Sendmail - Message transfer agent (MTA).
Operating system–level virtualization.
- Bitnami - Produces open source installers or software packages for web applications and development stacks as well as virtual appliances.
- Docker - Open platform for developers and sysadmins to build, ship, and run distributed applications.
- OpenVZ - Container-based virtualization for Linux.
SSH tools.
- autossh - Automatically respawn ssh session after network interruption.
- Cluster SSH - Controls a number of xterm windows via a single graphical console.
- DSH - Dancer's shell / distributed shell - Wrapper for executing multiple remote shell commands from one command line.
- Mosh - The mobile shell.
- parallel-ssh - Provides parallel versions of OpenSSH and related tools.
- SSH Power Tool - Execute commands and upload files to many servers simultaneously without using pre-shared keys.
Analytics software.
- Analog - The most popular logfile analyser in the world.
- GoAccess - Open source real-time web log analyzer and interactive viewer that runs in a terminal.
- Piwik - Free and open source web analytics application.
- Webalizer - Fast, free web server log file analysis program.
Web-based ticketing system.
- Bugzilla - General-purpose bugtracker and testing tool originally developed and used by the Mozilla project.
- Cerb - A group-based e-mail management project built with a commercial open source license.
- Flyspray - Web-based bug tracking system written in PHP.
- MantisBT - Another web-based bug tracking system.
- osTicket - Open source support ticket system.
- Otrs - A free and open-source trouble ticket system software package that a company, organization, or other entity can use to assign tickets to incoming queries and track further communications about them.
- Request Tracker - Ticket-tracking system written in Perl.
- TheBugGenie - Open source ticket system with extremely complete users rights granularity.
Troubleshooting Tools.
- mitmproxy - A Python tool used for intercepting, viewing and modifying network traffic. Invaluable in troubleshooting certain problems.
- Sysdig - Capture system state and activity from a running Linux instance, then save, filter and analyze.
- Wireshark - The world's foremost network protocol analyzer.
Web-based project management and bug tracking systems.
- ChiliProject - Fork of Redmine.
- GitBucket Clone of GitHub written in Scala; single jar install.
- GitLab - Clone of GitHub written in Ruby.
- Gogs - Written in Go.
- OpenProject - Project collaboration with open source.
- Phabricator Written in PHP.
- Redmine - Written in ruby on rails.
- The Bug Genie - Written in PHP.
- Trac - Written in python.
Software versioning and revision control.
- Fossil - Distributed version control with built-in wiki and bug tracking.
- Git - Distributed revision control and source code management (SCM) with an emphasis on speed.
- GNU Bazaar - Distributed revision control system sponsored by Canonical.
- Mercurial - Another distributed revision control.
- Subversion - Client-server revision control system.
Virtualization software.
- Ganeti - Cluster virtual server management software tool built on top of KVM and Xen.
- KVM - Linux kernel virtualization infrastructure.
- oVirt - Manages virtual machines, storage and virtual networks.
- Packer - A tool for creating identical machine images for multiple platforms from a single source configuration.
- Vagrant - Tool for building complete development environments.
- VirtualBox - Virtualization product from Oracle Corporation.
- Xen - Virtual machine monitor for 32/64 bit Intel / AMD (IA 64) and PowerPC 970 architectures.
VPN software.
- OpenVPN - Uses a custom security protocol that utilizes SSL/TLS for key exchange.
- Pritunl - OpenVPN based solution. Easy to set up.
- SoftEther - Multi-protocol software VPN with advanced features
- sshuttle - Poor man's VPN.
- strongSwan - Complete IPsec implementation for Linux.
- tinc - Distributed p2p VPN.
XMPP servers.
- ejabberd - XMPP instant messaging server written in Erlang/OTP.
- Metronome IM - Fork of Prosody IM.
- MongooseIM - Fork of ejabberd.
- Openfire - Real time collaboration (RTC) server.
- Prosody IM - XMPP server written in Lua.
- Tigase - XMPP server implementation in Java.
Webmail applications.
- Mailpile - A modern, fast web-mail client with user-friendly encryption and privacy features.
- Roundcube - Browser-based IMAP client with an application-like user interface.
Web servers.
- Apache - Most popular web server.
- Cherokee - Lightweight, high-performance web server/reverse proxy.
- Lighttpd - Web server more optimized for speed-critical environments.
- Nginx - Reverse proxy, load balancer, HTTP cache, and web server.
- uWSGI - The uWSGI project aims at developing a full stack for building hosting services.
Web Performance
- HAProxy - Software based load Balancing, SSL offloading and performance optimization, compression, and general web routing.
- Varnish - HTTP based web application accelerator focusing on optimizing caching and compression.
Wiki software.
- DokuWiki - Simple to use and highly versatile wiki that doesn't require a database.
- Gollum - A simple, Git-powered wiki with a sweet API and local frontend.
- ikiwiki - A wiki compiler.
- Mediawiki - Used to power Wikipedia.
- MoinMoin - An advanced, easy to use and extensible WikiEngine with a large community of users.
- Ōlelo Wiki - A a wiki that stores pages in a Git repository.
- TiddlyWiki - Complete interactive wiki in JavaScript.
Various resources, such as books, websites and articles, for improving your skills and knowledge.
- Code as Craft - Etsy's Ops blog, lots of technical posts.
- DevOpsGuys - Devops consultants who blog about operations.
- Rackspace Developers - Slightly biased blog with lots of Devops Topics.
Sysadmin related books.
- The Linux Command Line - A book about the Linux command line by William Shotts.
- The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win - How DevOps techniques can fix the problems that happen in IT organizations.
- The Practice of System and Network Administration - The first and second editions describes the best practices of system and network administration, independent of specific platforms or technologies.
- The Visible Ops Handbook: Implementing ITIL in 4 Practical and Auditable Steps - Is a methodology designed to jumpstart implementation of controls and process improvement.
- UNIX and Linux System Administration Handbook - Approaches system administration from a practical perspective.
Open source code editors.
- Atom - A hackable text editor from Github.
- Brackets - Open source code editor for web designers and front-end developers.
- Eclipse - IDE written in Java with an extensible plug-in system.
- Geany - GTK2 text editor.
- GNU Emacs - An extensible, customizable text editor-and more.
- Haroopad - Markdown editor with live preview.
- ICEcoder - Code editor awesomeness, built with common web languages.
- jotgit - Git-backed real-time collaborative code editing.
- Light Table - The next generation code editor.
- Lime - Aims to provide an open source solution to Sublime Text
- TextMate - A graphical text editor for OS X.
- Vim - A highly configurable text editor built to enable efficient editing.
- Servers for Hackers - Newsletter for programmers who find themselves needing to know their way around a server.
Software package repositories.
- Dotdeb - Repository with LAMP updated packages for Debian.
- Remi - Repository with LAMP updated packages for RHEL/Centos/Fedora.
Useful sysadmin related websites.
- Ops School - Comprehensive program that will help you learn to be an operations engineer.
- Digital Ocean Tutorials - A surprisingly vast resource for getting the basics of certain applications, tools, or even systems administration topics.
Please see CONTRIBUTING for details.