Skip to main content

Posts

Showing posts from December, 2020

The meaning of time in reinforcement learning

Reinforcement learning (RL) is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. Reinforcement learning is concerned with how software agents ought to take actions in an environment in order to maximize the notion of cumulative reward through the process of trial and error. In reinforcement learning an agent starts at an empty state then analyzes the available datasets according to a policy of positive states and negative states. Rather than being explicitly taught as in supervised learning the correct set of actions for performing a task, reinforcement learning uses rewards as signals for positive states and punishments as signals for negative states. The agent obtains the best path to a desirable reward as a cumulation of positive states and negative states. As compared to unsupervised learning, reinforcement learning is different in terms of goals. While the goal in unsupervised learning is to find similarities and differences

Web app comparison of async and real-time

Advantages of asynchronous web apps Generic request/response structure Stateless session control Message queue management Token access based on serverside date-time format PostgreSQL paging using token-centric tables and functions Shared pools of resources per customer One-to-many security policies Single domain name with TLS cert over HTTPS Shared bandwidth for uploads/downloads No endpoint/device registration No direct access to server resources Web app aggregation as control panel A/B Testing Advantages of (near) real-time web apps Stateful session control On-demand communication protocols per customer Custom request/response structure per customer Custom date-time formats per customer Endpoint/device registration PostgreSQL paging using static tables and aggregate functions Immediate execution of requests Dedicated pools of resources per customer Dedicated TLS cert over HTTPS per customer Dedicated IP addres

OpenStack+Ceph as Software-Defined Storage

SDS reduces the costs of the management of growing data stores by decoupling storage management from its hardware to allow for centralized management of cheaper, popular commodity hardware. The example SDS ecosystem uses open source software like OpenStack as a front-end interface on top of Ceph as the resource provider of a RADOS cluster of commodity solid-state drives. OpenStack provides user-friendly wrappers for accessing and modifying underlying Ceph storage. OpenStack comes in the form of distributed microservices with RESTful API's: Block (Cinder), File (Manila), Image (Glance), and Object (Swift). Each microservice can scale-out as a cluster of stand-alone services to accommodate the varying demands of high-growth storage. With OpenStack the underlying Ceph storage can address the block storage needs, file storage needs, image storage needs, and object storage needs of datacenters adopting open source as their new norm in an industry trend for high performace and high a