Async/src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php

384 lines
13 KiB
PHP
Raw Normal View History

<?php
/**
* FuzeWorks Async Library
*
* The FuzeWorks PHP FrameWork
*
* Copyright (C) 2013-2020 TechFuze
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
* @author TechFuze
* @copyright Copyright (c) 2013 - 2020, TechFuze. (http://techfuze.net)
* @license https://opensource.org/licenses/MIT MIT License
*
* @link http://techfuze.net/fuzeworks
* @since Version 1.0.0
*
* @version Version 1.0.0
*/
namespace FuzeWorks\Async\TaskStorage;
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
use FuzeWorks\Async\Events\TaskModifyEvent;
use FuzeWorks\Async\Task;
use FuzeWorks\Async\TasksException;
use FuzeWorks\Async\TaskStorage;
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
use FuzeWorks\Events;
use FuzeWorks\Logger;
use Redis;
use RedisException;
class RedisTaskStorage implements TaskStorage
{
/**
* @var Redis
*/
protected $conn;
protected $indexSet = 'async_index';
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
protected $unfinishedSet = 'async_index_unfinished';
protected $key_prefix = 'async_task_';
/**
* @inheritDoc
*/
public function __construct(array $parameters)
{
// Attempt to connect to Redis
try {
$this->conn = new Redis();
// Afterwards connect to server
$socketType = $parameters['socket_type'];
if ($socketType == 'unix')
$success = $this->conn->connect($parameters['socket']);
elseif ($socketType == 'tcp')
$success = $this->conn->connect($parameters['host'], $parameters['port'], $parameters['timeout']);
else
$success = false;
// If unsuccessful, return false
if (!$success)
throw new TasksException("Could not construct RedisTaskStorage. Failed to connect.");
// Otherwise attempt authentication, if needed
if (isset($parameters['password']) && !$this->conn->auth($parameters['password']))
throw new TasksException("Could not construct RedisTaskStorage. Authentication failure.");
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// And select the DB index
$this->conn->select($parameters['db_index']);
} catch (RedisException $e) {
throw new TasksException("Could not construct RedisTaskStorage. RedisException thrown: '" . $e->getMessage() . "'");
}
}
/**
* @inheritDoc
* @throws TasksException
*/
public function addTask(Task $task): bool
{
// Check if the task doesn't exist yet
$taskId = $task->getId();
// Query the index
$isMember = $this->conn->sIsMember($this->indexSet, $taskId);
if ($isMember)
throw new TasksException("Could not add Task to TaskStorage. Task '$taskId' already exists.");
// Serialize the task and save it
$taskData = serialize($task);
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// Register the task
$this->conn->sAdd($this->indexSet, $taskId);
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
if ($task->getStatus() !== Task::COMPLETED && $task->getStatus() !== Task::CANCELLED)
$this->conn->sAdd($this->unfinishedSet, $taskId);
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// And create a hash for it
if ($this->conn->hSet($this->key_prefix . $taskId, 'data', $taskData) === FALSE)
return false;
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
return true;
}
/**
* @inheritDoc
*/
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
public function readTasks(bool $noIncludeDone = false): array
{
// First fetch an array of all tasks in the set
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
if ($noIncludeDone)
$taskList = $this->conn->sMembers($this->unfinishedSet);
else
$taskList = $this->conn->sMembers($this->indexSet);
// Go over each taskId and fetch the specific task
$tasks = [];
foreach ($taskList as $taskId)
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
$tasks[] = unserialize($this->conn->hGet($this->key_prefix . $taskId, 'data'));
return $tasks;
}
/**
* @inheritDoc
*/
public function getTaskById(string $identifier): Task
{
// Query the index
$isMember = $this->conn->sIsMember($this->indexSet, $identifier);
if (!$isMember)
throw new TasksException("Could not get task by ID. Task not found.");
// Fetch the task
/** @var Task $task */
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
$task = unserialize($this->conn->hGet($this->key_prefix . $identifier, 'data'));
// Return the task
return $task;
}
/**
* @inheritDoc
*/
public function modifyTask(Task $task): bool
{
// First get the task ID
$taskId = $task->getId();
// Check if it exists
$isMember = $this->conn->sIsMember($this->indexSet, $taskId);
if (!$isMember)
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
throw new TasksException("Could not modify task. Task '$taskId' does not exists.");
// Fire the TaskModifyEvent
/** @var TaskModifyEvent $event */
$event = Events::fireEvent(new TaskModifyEvent(), $task);
if ($event->isCancelled())
{
Logger::log("Did not modify task. Cancelled by taskModifyEvent.");
return false;
}
// And write the data
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
$taskData = serialize($event->getTask());
if ($this->conn->hSet($this->key_prefix . $taskId, 'data', $taskData) === FALSE)
return false;
// Modify the unfinished set
if ($this->conn->sIsMember($this->unfinishedSet, $taskId) && ($task->getStatus() === Task::COMPLETED || $task->getStatus() === Task::CANCELLED ))
$this->conn->sRem($this->unfinishedSet, $taskId);
elseif (!$this->conn->sIsMember($this->unfinishedSet, $taskId) && $task->getStatus() !== Task::COMPLETED && $task->getStatus() !== Task::CANCELLED)
$this->conn->sAdd($this->unfinishedSet, $taskId);
return true;
}
/**
* @inheritDoc
* @throws TasksException
*/
public function deleteTask(Task $task): bool
{
// First get the task ID
$taskId = $task->getId();
// Check if it exists
$isMember = $this->conn->sIsMember($this->indexSet, $taskId);
if (!$isMember)
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
throw new TasksException("Could not delete task. Task '$taskId' does not exists.");
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// Delete the task from the index
$this->conn->sRem($this->indexSet, $taskId);
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// And remove the task from the unfinishedSet
if ($this->conn->sIsMember($this->unfinishedSet, $taskId))
$this->conn->sRem($this->unfinishedSet, $taskId);
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// Delete the task itself
if ($this->conn->del($this->key_prefix . $taskId) > 0)
return true;
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
return false;
}
/**
* @inheritDoc
*/
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
public function writeTaskOutput(Task $task, string $output, string $errors, int $statusCode): bool
{
// First get the task ID
$taskId = $task->getId();
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// Check if the task exists
$isMember = $this->conn->sIsMember($this->indexSet, $taskId);
if (!$isMember)
throw new TasksException("Could not write task output. Task '$taskId' not found.");
// Prepare the data
$contents = ['taskId' => $taskId, 'output' => $output, 'errors' => $errors, 'statusCode' => $statusCode];
// Determine the attempt number
$attempt = $this->conn->hIncrBy($this->key_prefix . $taskId, 'taskOutputAttempts', 1);
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// Then write this output
if ($this->conn->hSet($this->key_prefix . $taskId, 'output' . $attempt, serialize($contents)) === FALSE)
return false;
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
return true;
}
/**
* @inheritDoc
*/
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
public function writePostOutput(Task $task, string $output, string $errors, int $statusCode): bool
{
// First get the task ID
$taskId = $task->getId();
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// Check if the task exists
$isMember = $this->conn->sIsMember($this->indexSet, $taskId);
if (!$isMember)
throw new TasksException("Could not write post output. Task '$taskId' not found.");
// Prepare the data
$contents = ['taskId' => $taskId, 'output' => $output, 'errors' => $errors, 'statusCode' => $statusCode];
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// Determine the attempt number
$attempt = $this->conn->hIncrBy($this->key_prefix . $taskId, 'taskPostAttempts', 1);
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// Then write this output
if ($this->conn->hSet($this->key_prefix . $taskId, 'postOutput' . $attempt, serialize($contents)) === FALSE)
return false;
return true;
}
/**
* @inheritDoc
*/
public function readTaskOutput(Task $task, int $attempt = 0): ?array
{
// First get the task ID
$taskId = $task->getId();
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// If a nothing in particular is requested, fetch the latest and select that as the attempt
if ($attempt === 0)
$attempt = $this->conn->hGet($this->key_prefix . $taskId, 'taskOutputAttempts');
// If -1 is requested, fetch all
if ($attempt === -1)
{
// Get amount of attempts
$totalAttempts = $this->conn->hGet($this->key_prefix . $taskId, 'taskOutputAttempts');
$output = [];
for ($i=1;$i<=$totalAttempts;$i++)
{
// Check if this output exists
if (!$this->conn->hExists($this->key_prefix . $taskId, 'output' . $i))
$output[] = null;
// Load and convert the data
$data = $this->conn->hGet($this->key_prefix . $taskId, 'output' . $i);
$data = unserialize($data);
$output[] = ['output' => $data['output'], 'errors' => $data['errors'], 'statusCode' => $data['statusCode']];
}
if (!empty($output))
return $output;
}
// If a specific one is requested, fetch that one
else
{
// Check if this output exists
if (!$this->conn->hExists($this->key_prefix . $taskId, 'output' . $attempt))
return null;
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// Load and convert the data
$data = $this->conn->hGet($this->key_prefix . $taskId, 'output' . $attempt);
$data = unserialize($data);
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
return ['output' => $data['output'], 'errors' => $data['errors'], 'statusCode' => $data['statusCode']];
}
return null;
}
/**
* @inheritDoc
*/
public function readPostOutput(Task $task, int $attempt = 0): ?array
{
// First get the task ID
$taskId = $task->getId();
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// If a nothing in particular is requested, fetch the latest and select that as the attempt
if ($attempt === 0)
$attempt = $this->conn->hGet($this->key_prefix . $taskId, 'taskPostAttempts');
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
// If -1 is requested, fetch all
if ($attempt === -1)
{
// Get amount of attempts
$totalAttempts = $this->conn->hGet($this->key_prefix . $taskId, 'taskPostAttempts');
$output = [];
for ($i=1;$i<=$totalAttempts;$i++)
{
// Check if this output exists
if (!$this->conn->hExists($this->key_prefix . $taskId, 'postOutput' . $i))
$output[] = null;
// Load and convert the data
$data = $this->conn->hGet($this->key_prefix . $taskId, 'postOutput' . $i);
$data = unserialize($data);
$output[] = ['output' => $data['output'], 'errors' => $data['errors'], 'statusCode' => $data['statusCode']];
}
if (!empty($output))
return $output;
}
// If a specific one is requested, fetch that one
else
{
// Check if this output exists
if (!$this->conn->hExists($this->key_prefix . $taskId, 'postOutput' . $attempt))
return null;
// Load and convert the data
$data = $this->conn->hGet($this->key_prefix . $taskId, 'postOutput' . $attempt);
$data = unserialize($data);
Release of RC1 (#7) Finished ControllerHandler into a working state. Merge remote-tracking branch 'origin/master' into 3-features # Conflicts: # Dockerfile # bin/supervisor # bin/worker # composer.json # src/FuzeWorks/Async/Executors/ShellExecutor.php # src/FuzeWorks/Async/ShellWorker.php # src/FuzeWorks/Async/Supervisors/ParallelSuperVisor.php # src/FuzeWorks/Async/TaskStorage/RedisTaskStorage.php # src/FuzeWorks/Async/Tasks.php # test/bootstrap.php # test/mock/Handlers/EmptyHandler.php # test/mock/Handlers/TestStopTaskHandler.php Started work on making tasks forcefully quit after a maximum time has expired. Finished tasks are no longer loaded in SuperVisor. By adding a parameter in TaskStorage, it is now possible to distinguish between finished and unfinished tasks. Finished tasks are those tasks that have a status of Task::COMPLETED or Task::CANCELLED. Unfinished tasks are all others. This allows the SuperVisor to not bother with the mountain of tasks that will be saved during large projects. Implemented proper dependencies. Dependencies can now pass output to each other using the DependentTaskHandler. Also fixed some general problems in the Tasks class, for instance the Executor not starting correctly because of problematic parameters. Also, SuperVisor now sets the output of a Task using the last output of the task, and not the first. Return all output when providing attempt = 0. When providing $attempt = 0 at readTaskOutput and readPostOutput, all output shall be returned. This is the default return. Hence, a lot of tests had to be altered slightly. Changed the way task output and post output is saved. Redis now saves all output for a task within a hash. This hash contains individual tasks and all its output. Attempts also start at 1, since that makes most sense for this context. When output is written, the TaskStorage must figure out at which attempt the Task is. Implemented Parent Handlers. Parent Handlers can be stacked to run in succession. Output is transfered as input into the child handler which can continue with it. If the parent Handler fails, all Child handlers also fail. Made Handlers an object instead of a string reference. Handlers should now be added as objects, adding some flexibility to the developer. Developers are still cautioned to take great care that Handlers work approriately. Handlers can potentially crash the SuperVisor if not taken good care of. Try with only Redis. Made many changes. Fixed race-conditions in test code. Now try while flushing a selected database. Try again in the new environment. Maybe Events are the problem? Fixed DummyTaskStorage persisting outside of the storage. Awkward how that could go wrong... Added TaskModifyEvent. Event gets fired when an Event is modified by sending it to TaskStorage::modifyEvent. This allows components to observe changes and report these to the user. Might also be useful to cancel unwanted changes. Made the Docker image Alpine-based. Should work better when running Async in a Cron environment. Also removed compatibility with PHP 7.2. Implemented many unit tests. Now with coverage And remove the Redis debug again Now? Temporarily check if Redis works Added separate environments for DummyTaskStorage and RedisTaskStorage. System now uses environment variables imported through Docker. See test/config.tasks.php for all environment options available. Now try with an added service Attempt to run a PHPUnit batch Try with a modified environment. Try again Started implementing Drone ControllerHandler now works. Next up is a distinction between Task and Process status. Started implementing ControllerHandler. ControllerHandler is a standard utility handler for MVCR Controllers. This allows the user to create a task which is handled by a standardized controller. Not finished yet! Needs some love. Add 'addTasks' method to `Tasks` class Implemented basic RedisTaskStorage. - Fixed bug where worker is not provided with bootstrap by ShellExecutor. - Added composer and Redis to Dockerfile - Added more output to ParallelSuperVisor Updated config format. Implemented changes to binaries. Binaries now accept a 'bootstrap' argument, allowing the developer to load a custom bootstrap from the project they're working on. This allows Async to run in the same environment as the project it's part of. Co-authored-by: Abel Hoogeveen <abel@techfuze.net> Reviewed-on: https://gitea.i15.nl/FuzeWorks/Async/pulls/7
2020-06-07 13:54:19 +00:00
return ['output' => $data['output'], 'errors' => $data['errors'], 'statusCode' => $data['statusCode']];
}
return null;
}
/**
* @inheritDoc
*/
public function reset(): bool
{
// Clear the current db
return $this->conn->flushDB();
}
}