浏览代码

[Frontend] Task server frontend changes. (#3673)

[Frontend] Added a task server page, tasks table. desktop document clean up works (have to move it to celery though). Need to add a django db table to store the tasks table browser

Change-Id: I661b26388ffca4c42963b3ffcdf20f13c9914fa6
(cherry picked from commit 9a17b999b09d2015608f3795e3a2f26765570dfd)

[Frontend] 1. Added a django db table.  2. React fetches the results from this db table. 3. Added migrations scripts for the newly created task-models.py - django table

Change-Id: I7ae105083c0a5bac884c59b34cd801f5dee23f83
(cherry picked from commit 2287ffe61fab8d18ded0e8d998efd5cc65ce8d0b)

[Frontend] Adding styling changes for task browser tables and adding label/icons

Change-Id: I3ee803fdaf60f808e5eb62151f15a94cc678561f
(cherry picked from commit 4d089f1828a4ce14004e4068286bcdcc4fcc245d)

[Frontend] Incomplete changes - greyout/reload file upload page, desktop_document_cleanup using redis

Change-Id: Ib53a9b81000a6c488b079bbcce617f8986ff3880
(cherry picked from commit 9fbdf55a302808d5a44a6a8a4311468f6617c92d)

[Frontend] Fixing the upload popup, the stay until all uploads are complete. Fixing document clean up install dir and duration calculation

Change-Id: Ibf28fc12a05d4160a83ae57f727735daa3d8d6e7
(cherry picked from commit 403e990bb166683eea28436a4bba301cb3f9295d)

[Frontend] schedule /tmp clean_up job in task_server page

Change-Id: Id9d4ec893674c5df350bce3dbb8d4d21c254d1ce
(cherry picked from commit 32332daf797a16b5c2409ea5335f6cc8307267aa)

[Frontend] schedule /tmp clean_up job in task_server page

Change-Id: I2c039e46c1b81a9fc2df55301b9d3378338fbc16
(cherry picked from commit 366bcaa9314ba53f7c393d87b835c41b8600c9a2)

[Frontend] Changing progress Filter buttons (success, failure, running) to checkboxes

Change-Id: Iddbf0a687623e0bd598d5acd85201ace97510b36
(cherry picked from commit 006e14b663ad40d323589f9b87e554cd237461c6)

[Frontend] Adding schedule upload support. Cleaned up default fallbackcomponent. Removed the csrf exempt annotations

Change-Id: I03b5ce1d0aba14fad50f0b41d14d71d1a4b8cf84
(cherry picked from commit 5d562a7b82664dc471d56303497e222db979ef92)

[Frontend] /tmp cleaner runs every 1000 seconds using celery-beat scheduler. Files written onto /tmp folder in the last 15 minutes wont be deleted. This time interval ensures that when a user/service is writing onto /tmp, the files are not deleted.

Change-Id: Ic6b31173cc9a778da0c055a18435bdafc7384141
(cherry picked from commit cb0d1a01b690232e5efc8ca91d322c77c2e225d3)

[Frontend] Adding support for killing celery tasks

Change-Id: Iba42d5f1d3c36dfddeeb8add2dcfa39b158b50d2
(cherry picked from commit 927d959ae82866240cf361ba63a747b45c0fc8bd)

[Frontend] Restricting uploads based on free space available in /tmp and MAX_FILE_SIZE_UPLOAD_LIMIT

Change-Id: I42d6d7a379dbd5b655be6b0e01c859d0b51294b3
(cherry picked from commit 8f4819dafb884300899855b0526e909a27a7da81)

[Frontend] Improving schedule task popup and adding default values for tasks

Change-Id: If2e5ac784d338f479715741fc2ea74d97ad48b53
(cherry picked from commit 3f63e1ee2bd0a34a0c569293c759a1807f93349b)

Improving styling using Antd and CUIX, pollForRealProgress. Cleaning up comments.

Change-Id: Ifdd00a48d366a0b845b0682f6d20df8f08f8e91f
(cherry picked from commit f7af6a64022b227849115326c6a91bf7fe33f0e9)

Improving styling using Antd and CUIX, pollForRealProgress. Cleaning up comments.

Change-Id: Ifdd00a48d366a0b845b0682f6d20df8f08f8e91f
(cherry picked from commit d01c62e555b842dfb26c1564fd75b88e8ca1f451)

Improving styling using Antd and CUIX, pollForRealProgress. Cleaning up comments.

Change-Id: Ifdd00a48d366a0b845b0682f6d20df8f08f8e91f
(cherry picked from commit ccf28baf174f93c1c762a767bc090c83b4084596)

Improving styling using Antd and CUIX, pollForRealProgress. Cleaning up comments.

Change-Id: Ifdd00a48d366a0b845b0682f6d20df8f08f8e91f
(cherry picked from commit e88f14cf0a141c76ae76c8e469b2e8372a64cdb4)

Only schedule upload button is enabled when task server is configured. Added tool tip for schedule upload button.

Change-Id: I4fb0dbd5626cd717ded3a1a3a1c40f0232b73067

improving log popup modal and url routing

Change-Id: Idb7637de564392142f6eb39bd58c5a8ee7fbe5dc

Co-authored-by: Athithyaa Selvam <aselvam@cloudera.com>
Athithyaa Selvam 1 年之前
父节点
当前提交
1cd19ea901

+ 22 - 3
apps/filebrowser/src/filebrowser/templates/listdir.mako

@@ -164,21 +164,40 @@ ${ fb_components.menubar() }
         <div class="btn-toolbar" style="display: inline; vertical-align: middle">
           % if show_upload_button:
           <!-- ko if: isS3 -->
-            <a class="btn fileToolbarBtn" title="${_('Upload files')}" data-bind="visible: !inTrash(), css: {'disabled': isS3Root()}, click: function(){ if (!isS3Root()) { uploadFile() }}"><i class="fa fa-arrow-circle-o-up"></i> ${_('Upload')}</a>
+            <!-- ko ifnot: isTaskServerEnabled -->
+            <a class="btn fileToolbarBtn" title="${_('Upload files')}" data-bind="visible: !inTrash(), css: {'disabled': isS3Root()}, click: function(){ if (!isS3Root()) { uploadFile(false) }}"><i class="fa fa-arrow-circle-o-up"></i> ${_('Upload')}</a>
+            <!-- /ko -->
+            <!-- ko if: isTaskServerEnabled -->
+            <a class="btn fileToolbarBtn" title="${_('Select a file to upload. The file will first be saved locally and then automatically transferred to the designated file system (e.g., S3, Azure) in the background. The upload modal closes immediately after the file is queued, allowing you to continue working. A notification, \'File upload scheduled. Please check the task server page for progress,\' will confirm the upload has started. This feature is especially useful for large files, as it eliminates the need to wait for the upload to complete.')}" data-bind="visible: !inTrash(), css: {'disabled': isS3Root()}, click: function(){ if (!isS3Root()) { uploadFile(true) }}"><i class="fa fa-arrow-circle-o-up"></i> ${_('Schedule Upload')}</a>
+            <!-- /ko -->
           <!-- /ko -->
           <!-- ko if: isGS -->
             <a class="btn fileToolbarBtn" title="${_('Upload files')}" data-bind="visible: !inTrash(), css: {'disabled': isGSRoot()}, click: function(){ if (!isGSRoot()) { uploadFile() }}"><i class="fa fa-arrow-circle-o-up"></i> ${_('Upload')}</a>
           <!-- /ko -->
           <!-- ko if: isABFS -->
-            <a class="btn fileToolbarBtn" title="${_('Upload files')}" data-bind="visible: !inTrash(), css: {'disabled': isABFSRoot()}, click: function(){ if (!isABFSRoot()) { uploadFile() }}"><i class="fa fa-arrow-circle-o-up"></i> ${_('Upload')}</a>
+            <!-- ko ifnot: isTaskServerEnabled -->
+            <a class="btn fileToolbarBtn" title="${_('Upload files')}" data-bind="visible: !inTrash(), css: {'disabled': isABFSRoot()}, click: function(){ if (!isABFSRoot()) { uploadFile(false) }}"><i class="fa fa-arrow-circle-o-up"></i> ${_('Upload')}</a>
+            <!-- /ko -->
+            <!-- ko if: isTaskServerEnabled -->
+            <a class="btn fileToolbarBtn" title="${_('Select a file to upload. The file will first be saved locally and then automatically transferred to the designated file system (e.g., S3, Azure) in the background. The upload modal closes immediately after the file is queued, allowing you to continue working. A notification, \'File upload scheduled. Please check the task server page for progress,\' will confirm the upload has started. This feature is especially useful for large files, as it eliminates the need to wait for the upload to complete.')}" data-bind="visible: !inTrash(), css: {'disabled': isABFSRoot()}, click: function(){ if (!isABFSRoot()) { uploadFile(true) }}"><i class="fa fa-arrow-circle-o-up"></i> ${_('Schedule Upload')}</a>
+            <!-- /ko -->
           <!-- /ko -->
           <!-- ko ifnot: isS3() || isGS() || isABFS() -->
+          <!-- ko ifnot: isTaskServerEnabled -->
           <div id="upload-dropdown" class="btn-group" style="vertical-align: middle">
-            <a data-hue-analytics="filebrowser:upload-btn-click" href="javascript: void(0)" class="btn upload-link dropdown-toggle" title="${_('Upload')}" data-bind="click: uploadFile, visible: !inTrash(), css: {'disabled': (isOFS() && (isOFSRoot() || isOFSServiceID() || isOFSVol()))}">
+            <a data-hue-analytics="filebrowser:upload-btn-click" href="javascript: void(0)" class="btn upload-link dropdown-toggle" title="${_('Upload')}" data-bind="click: function() { uploadFile(false); }, visible: !inTrash(), css: {'disabled': (isOFS() && (isOFSRoot() || isOFSServiceID() || isOFSVol()))}">
               <i class="fa fa-arrow-circle-o-up"></i> ${_('Upload')}
             </a>
           </div>
           <!-- /ko -->
+          <!-- ko if: isTaskServerEnabled -->
+          <div id="upload-dropdown" class="btn-group" style="vertical-align: middle">
+            <a data-hue-analytics="filebrowser:upload-btn-click" href="javascript: void(0)" class="btn upload-link dropdown-toggle" title="${_('Select a file to upload. The file will first be saved locally and then automatically transferred to the designated file system (e.g., S3, Azure) in the background. The upload modal closes immediately after the file is queued, allowing you to continue working. A notification, \'File upload scheduled. Please check the task server page for progress,\' will confirm the upload has started. This feature is especially useful for large files, as it eliminates the need to wait for the upload to complete.')}" data-bind="click: function() { uploadFile(true); }, visible: !inTrash(), css: {'disabled': (isOFS() && (isOFSRoot() || isOFSServiceID() || isOFSVol()))}">
+              <i class="fa fa-arrow-circle-o-up"></i> ${_('Schedule Upload')}
+            </a>
+          </div>
+          <!-- /ko -->
+          <!-- /ko -->
           % endif
           <div class="btn-group" style="vertical-align: middle">
             <a href="javascript: void(0)" data-toggle="dropdown" class="btn dropdown-toggle" data-bind="visible: !inTrash(), css: {'disabled': isOFSRoot()}">

+ 163 - 6
apps/filebrowser/src/filebrowser/templates/listdir_components.mako

@@ -22,7 +22,7 @@ from django.template.defaultfilters import urlencode, stringformat, filesizeform
 from desktop.lib.django_util import reverse_with_get, extract_field_data
 from django.utils.encoding import smart_str
 
-from filebrowser.conf import ENABLE_EXTRACT_UPLOADED_ARCHIVE, FILE_UPLOAD_CHUNK_SIZE, CONCURRENT_MAX_CONNECTIONS
+from filebrowser.conf import ENABLE_EXTRACT_UPLOADED_ARCHIVE, FILE_UPLOAD_CHUNK_SIZE, CONCURRENT_MAX_CONNECTIONS, MAX_FILE_SIZE_UPLOAD_LIMIT
 
 if sys.version_info[0] > 2:
   from django.utils.translation import gettext as _
@@ -1109,6 +1109,10 @@ else:
         return self.currentPath().toLowerCase().indexOf('ofs://') === 0;
       });
 
+      self.isTaskServerEnabled = ko.computed(function() {
+        return window.getLastKnownConfig().hue_config.enable_chunked_file_uploader && window.getLastKnownConfig().hue_config.enable_task_server;
+      });
+
       self.scheme = ko.pureComputed(function () {
         var path = self.currentPath();
         return path.substring(0, path.indexOf(':/')) || "hdfs";
@@ -2061,9 +2065,161 @@ else:
         });
       };
 
-      self.uploadFile = (function () {
-          var uploader;  
-          if (window.getLastKnownConfig().hue_config.enable_chunked_file_uploader) {
+      function pollForTaskProgress(taskId, listItem, fileName) {
+        var taskStatus = 'pending';
+        var pollingInterval = 10000;
+
+        var doPoll = function() {
+          if (taskStatus === 'pending') {
+            $.get('/desktop/api2/taskserver/check_upload_status/' + taskId, function(data) {
+              if (data.isFinalized || data.isFailure || data.is_revoked) {
+                taskStatus = data.isFinalized ? 'finalized' : 'failed';
+
+                if (data.isFinalized) {
+                  listItem.find('.progress-row-bar').css('width', '100%');
+                  listItem.find('.progress-row-text').text('Upload complete.');
+                  $(document).trigger('info', fileName + "${ _(' uploaded successfully.') }");
+                  self.retrieveData(true);
+                } else if (data.isFailure) {
+                  listItem.find('.progress-row-bar').css('width', '100%');
+                  listItem.find('.progress-row-text').text('Upload failed.');
+                  $(document).trigger('error', fileName + "${ _(' file upload failed. Please check the logs for task id: ') }" + taskId);
+                }
+                
+              } else if (data.isRunning) {
+                var progressPercentage = 90; // Adjust based on data.progress if available
+                listItem.find('.progress-row-bar').css('width', progressPercentage + '%');
+                setTimeout(doPoll, pollingInterval);
+              }
+            }).fail(function(xhr, textStatus, errorThrown) {
+              if (xhr.status === 404) {
+                setTimeout(doPoll, pollingInterval); // Retry after 10 seconds
+              }
+            });
+          }
+        };
+        self.retrieveData(true);
+        doPoll();
+      }
+
+      self.uploadFile = (function () {  
+          var uploader; 
+          var scheduleUpload;
+          
+          if ((window.getLastKnownConfig().hue_config.enable_chunked_file_uploader) && (window.getLastKnownConfig().hue_config.enable_task_server))  {
+            
+            self.pendingUploads(0);
+            var action = "/filebrowser/upload/chunks/";
+            self.taskIds = [];
+            self.listItems = [];
+            uploader = new qq.FileUploader({
+              element: document.getElementById("fileUploader"),
+              request: {
+                  endpoint: action,
+                  paramsInBody: false,
+                  params: {
+                      dest: self.currentPath(),
+                      inputName: "hdfs_file"
+                  }
+              },
+              maxConnections: window.CONCURRENT_MAX_CONNECTIONS || 5,
+              chunking: {
+                  enabled: true,
+                  concurrent: {
+                      enabled: true
+                  },
+                  partSize: window.FILE_UPLOAD_CHUNK_SIZE || 5242880,
+                  success: {
+                      endpoint: "/filebrowser/upload/complete/"
+                  },
+                  paramNames: {
+                      partIndex: "qqpartindex",
+                      partByteOffset: "qqpartbyteoffset",
+                      chunkSize: "qqchunksize",
+                      totalFileSize: "qqtotalfilesize",
+                      totalParts: "qqtotalparts"
+                  }
+              },
+
+              template: 'qq-template',
+              callbacks: {
+                  onProgress: function (id, fileName, loaded, total) {
+
+                    $('.qq-upload-files').find('li').each(function(){
+                      var listItem = $(this);
+                      if (listItem.find('.qq-upload-file-selector').text() == fileName){
+                        //cap the progress at 80%
+                        listItem.find('.progress-row-bar').css('width', (loaded/total)*80 + '%');
+                        if ((loaded/total) === 80) {
+                          listItem.find('.progress-row-text').text('Finalizing upload...');
+                        }
+                      }
+                    });            
+                  },
+
+                  onComplete: function (id, fileName, response) {
+                    self.pendingUploads(self.pendingUploads() - 1);
+                    if (response.status != 0) {
+                      huePubSub.publish('hue.global.error', {message: "${ _('Error: ') }" + response.data});
+                    }
+                    else {
+                      var task_id = response.task_id;
+                      self.taskIds.push(task_id);
+                      var listItem = $('.qq-upload-files').find('li').filter(function() {
+                        return $(this).find('.qq-upload-file-selector').text() === fileName;
+                      });
+                      self.listItems.push(listItem);
+                        if (scheduleUpload && self.pendingUploads() === 0) {
+                          $('#uploadFileModal').modal('hide');
+                          $(document).trigger('info', "File upload scheduled. Please check the task server page for progress.");
+                        }
+                        pollForTaskProgress(response.task_id, listItem, fileName);
+                      self.filesToHighlight.push(response.path);                       
+                    }
+                    if (self.pendingUploads() === 0) {                    
+                      self.taskIds=[];
+                      self.listItems=[];
+                      self.retrieveData(true);
+                    }
+                  },
+                  onSubmit: function (id, fileName, responseJSON) {
+                      var deferred = new qq.Promise(); // Create a promise to defer the upload
+                      var uploader = this;
+                      
+                      // Make an AJAX request to check available disk space
+                      $.ajax({
+                        url: '/desktop/api2/taskserver/get_available_space/',
+                        success: function(response) {
+                          var freeSpace = response.free_space;
+                          var file = uploader.getFile(id); // Use the stored reference
+                          
+                          if ((file.size > freeSpace) && (file.size > window.MAX_FILE_SIZE_UPLOAD_LIMIT*1024*1024)) {
+                            $(document).trigger('info', "Not enough space available to upload this file.")
+                            deferred.failure(); // Reject the promise to cancel the upload
+                          } else {
+                            var newPath = "/filebrowser/upload/chunks/file?dest=" + encodeURIComponent(self.currentPath().normalize('NFC'));
+                            uploader.setEndpoint(newPath);
+                            self.pendingUploads(self.pendingUploads() + 1);
+                            deferred.success(); // Resolve the promise to allow the upload
+                          }
+                        },
+                        error: function(xhr, status, error) {
+                          alert('Error checking available space: ' + error);
+                          deferred.failure(); // Reject the promise to cancel the upload
+                        }
+                      });
+
+                      return deferred; // Return the promise to Fine Uploader
+                  },
+                  onCancel: function (id, fileName) {
+                    self.pendingUploads(self.pendingUploads() - 1);
+                  }
+              },
+              debug: false
+            });
+          }
+          // Chunked Fileuploader without Taskserver
+          else if ((window.getLastKnownConfig().hue_config.enable_chunked_file_uploader) && !(window.getLastKnownConfig().hue_config.enable_task_server)) {
             self.pendingUploads(0);
             var action = "/filebrowser/upload/chunks/";
             uploader = new qq.FileUploader({
@@ -2098,7 +2254,6 @@ else:
               template: 'qq-template',
               callbacks: {
                   onProgress: function (id, fileName, loaded, total) {
-                  console.log(loaded);
                   $('.qq-upload-files').find('li').each(function(){
                     var listItem = $(this);
                     if (listItem.find('.qq-upload-file-selector').text() == fileName){
@@ -2137,6 +2292,7 @@ else:
               debug: false
             });
           }
+          //Regular fileuploads
           else {
             self.pendingUploads(0);
             var action = "/filebrowser/upload/file";
@@ -2211,7 +2367,8 @@ else:
           });
         });
 
-        return function () {
+        return function (isScheduled) {
+          scheduleUpload = isScheduled;
           $("#uploadFileModal").modal({
             show: true
           });

+ 6 - 0
desktop/core/src/desktop/js/onePageViewModel.js

@@ -558,6 +558,12 @@ class OnePageViewModel {
           });
         }
       },
+      {
+        url: '/task_server',
+        app: function () {
+          self.loadApp('taskserver');
+        }
+      },
       {
         url: '/desktop/connectors',
         app: function () {

+ 1 - 1
desktop/core/src/desktop/js/reactComponents/FallbackComponent.jsx

@@ -12,4 +12,4 @@ const FallbackComponent = () => {
   );
 };
 
-export default FallbackComponent;
+export default FallbackComponent;

+ 108 - 0
desktop/core/src/desktop/js/reactComponents/TaskBrowser/TaskBrowser.scss

@@ -0,0 +1,108 @@
+@use 'variables' as vars;
+@import '@cloudera/cuix-core/variables.scss';
+
+.log-content-scrollable {
+  max-height: 600px;
+  overflow-y: auto;
+  margin-right: -16px;
+  padding-right: 16px;
+}
+
+.customCursorClass {
+  cursor: pointer;
+}
+
+.flex-container-style {
+  display: flex;
+  align-items: center;
+  gap: 10px;
+}
+
+.task-selection {
+  margin-bottom: 2rem;
+}
+
+.task-dropdown {
+  width: 100%;
+  padding: 0.5rem;
+  font-size: 1rem;
+}
+
+.parameter-inputs {
+  display: flex;
+  flex-direction: column;
+  gap: 1rem;
+  margin-top: 2rem;
+}
+
+.parameter-row {
+  display: flex;
+  align-items: center;
+}
+
+.parameter-label {
+  flex: 1;
+  width: 100px;
+  font-weight: bold;
+  margin-right: 1rem;
+}
+
+.parameter-input {
+  padding: 0.5rem;
+  flex: 1;
+  font-size: 1rem;
+  width: 100%;
+}
+
+.vertical-spacer {
+  height: 2rem;
+}
+
+.row-success {
+  border-left: 4px solid $fluidx-green-500;
+}
+
+.row-failure {
+  border-left: 4px solid $fluidx-red-500;
+}
+
+.row-running {
+  border-left: 4px solid $fluidx-amber-500;
+}
+
+.ant-btn-primary {
+  background-color: $fluidx-blue-500;
+  border-color: $fluidx-blue-600;
+}
+
+.ant-btn-danger {
+  background-color: $fluidx-red-500;
+  border-color: $fluidx-red-600;
+}
+
+.ant-tag-success {
+  background-color: $fluidx-green-500;
+  color: $fluidx-gray-900;
+}
+
+.ant-tag-error {
+  background-color: $fluidx-red-500;
+  color: $fluidx-gray-900;
+}
+
+.ant-tag-warning {
+  background-color: $fluidx-amber-500;
+  color: $fluidx-gray-900;
+}
+
+.success-text {
+  color: $fluidx-green-500;
+}
+
+.running-text {
+  color: $fluidx-amber-500;
+}
+
+.failed-text {
+  color: $fluidx-red-500;
+}

+ 501 - 0
desktop/core/src/desktop/js/reactComponents/TaskBrowser/TaskBrowser.tsx

@@ -0,0 +1,501 @@
+import React, { useState, useEffect } from 'react';
+import { act } from '@testing-library/react';
+import { i18nReact } from '../../../js/utils/i18nReact';
+import axios from 'axios';
+import { post, get, extractErrorMessage } from '../../../js/api/utils.ts';
+import { calculateDuration, formatTimestamp } from '../../../js/utils/dateTimeUtils.ts';
+import './TaskBrowser.scss';
+import huePubSub from '../../../js/utils/huePubSub';
+import Modal from 'antd/lib/modal/Modal';
+import { Button, Tag, Input, Checkbox, Form, Select } from 'antd';
+import 'antd/dist/antd.css';
+import PrimaryButton from 'cuix/dist/components/Button/PrimaryButton';
+import DangerButton from 'cuix/dist/components/Button/DangerButton';
+import Table from 'cuix/dist/components/Table';
+
+const { Option } = Select;
+
+const tasks = {
+  'document cleanup': {
+    params: [{ name: 'keep-days', type: 'integer' }]
+  },
+  'tmp clean up': {
+    params: [{ name: 'threshold for clean up', type: 'integer', max: 100 }]
+  }
+};
+
+const ShowScheduleTaskPopup = ({ onClose, onSubmit, open }) => {
+  const [selectedTask, setSelectedTask] = useState('');
+  const [params, setParams] = useState({
+    'keep-days': '30', // Default value for "keep days"
+    'threshold for clean up': '90' // Default value for "threshold for clean up"
+  });
+
+  const handleChange = e => {
+    const { name, value } = e.target;
+    setParams(prevParams => ({
+      ...prevParams,
+      [name]: value
+    }));
+  };
+
+  const handleSubmit = () => {
+    onSubmit(selectedTask, params);
+    onClose();
+  };
+
+  return (
+    <Modal
+      title={'Schedule Task'}
+      open={open}
+      onOk={handleSubmit}
+      onCancel={onClose}
+      okText={'Submit'}
+      width={530}
+      className="hue-file-chooser__modal"
+    >
+      <div className="task-selection">
+        <Select
+          value={selectedTask}
+          onChange={setSelectedTask}
+          placeholder="Select Task"
+          style={{ width: '100%' }}
+        >
+          {Object.keys(tasks).map(taskName => (
+            <Option key={taskName} value={taskName}>
+              {taskName}
+            </Option>
+          ))}
+        </Select>
+      </div>
+      <div className="vertical-spacer"></div>
+      {selectedTask && (
+        <div className="parameter-inputs">
+          {tasks[selectedTask].params.map(param => (
+            <div key={param.name} className="parameter-row">
+              <Form.Item label={param.name}>
+                <Input
+                  name={param.name}
+                  type="text"
+                  placeholder={param.name}
+                  onChange={handleChange}
+                  value={params[param.name] || ''}
+                />
+              </Form.Item>
+            </div>
+          ))}
+        </div>
+      )}
+    </Modal>
+  );
+};
+
+const TaskBrowser: React.FC = (): React.ReactElement => {
+  const [showSchedulePopup, showScheduleTaskPopup] = useState(false);
+  const [scheduledTasks] = useState([]); // Store an array of scheduled tasks
+  const [showLogPopup, setShowLogPopup] = useState(false); // This should match where you use it
+  const [, setCurrentTaskId] = useState(null);
+  const [taskLogs, setTaskLogs] = useState('');
+
+  const handleScheduleSubmit = (taskName, taskParams) => {
+    const payload = {
+      taskName,
+      taskParams
+    };
+
+    axios
+      .post('/desktop/api2/taskserver/handle_submit', payload, {
+        transformResponse: [
+          function (data) {
+            try {
+              return JSON.parse(data);
+            } catch (e) {
+              return data;
+            }
+          }
+        ]
+      })
+      .then(() => {
+        act(() => huePubSub.publish('hue.global.info', { message: `Task submitted successfully` }));
+      })
+      .catch(error => {
+        const errorMessage = extractErrorMessage(error);
+        act(() =>
+          huePubSub.publish('hue.global.error', {
+            message: `Failed to submit scheduling task ${errorMessage}`
+          })
+        );
+      });
+  };
+
+  // This function will open the schedule task popup
+  const handleSchedulePopup = () => {
+    showScheduleTaskPopup(true);
+  };
+
+  const ShowTaskLogsPopup = ({ logs, onClose, open }) => {
+    return (
+      <Modal
+        title="Task Logs"
+        visible={open}
+        onOk={onClose}
+        onCancel={onClose}
+        width={830}
+        footer={[
+          <Button key="back" onClick={onClose}>
+            Close
+          </Button>
+        ]}
+      >
+        <div className="log-content-scrollable">
+          {' '}
+          <pre>{logs}</pre>
+        </div>
+      </Modal>
+    );
+  };
+
+  const ShowTaskLogsHandler = taskId => {
+    setCurrentTaskId(taskId);
+    get(`/desktop/api2/taskserver/get_task_logs/${taskId}/`)
+      .then(response => {
+        setTaskLogs(response);
+        setShowLogPopup(true);
+      })
+      .catch(error => {
+        act(() =>
+          huePubSub.publish('hue.global.error', { message: `Error fetching task logs: ${error}` })
+        );
+      });
+  };
+
+  return (
+    <div>
+      {showSchedulePopup && (
+        <ShowScheduleTaskPopup
+          onClose={() => showScheduleTaskPopup(false)}
+          onSubmit={handleScheduleSubmit}
+          open={showSchedulePopup}
+        />
+      )}
+      <div style={{ margin: '20px 0' }}>
+        <TaskBrowserTable
+          tasks={scheduledTasks}
+          ShowTaskLogsHandler={ShowTaskLogsHandler}
+          handleSchedulePopup={handleSchedulePopup}
+        />
+      </div>
+      {showLogPopup && (
+        <ShowTaskLogsPopup
+          logs={taskLogs}
+          open={showLogPopup}
+          onClose={() => setShowLogPopup(false)}
+        />
+      )}
+    </div>
+  );
+};
+
+interface TaskBrowserTableProps {
+  ShowTaskLogsHandler: (taskId: string) => void;
+  handleSchedulePopup: () => void;
+}
+
+export const TaskBrowserTable: React.FC<TaskBrowserTableProps> = ({
+  ShowTaskLogsHandler,
+  handleSchedulePopup
+}) => {
+  const [tasks, setTasks] = useState<Task_Status[]>([]);
+  const [searchTerm, setSearchTerm] = useState(''); // state for the search term
+  const [statusFilter, setStatusFilter] = useState({
+    success: false,
+    failure: false,
+    running: false,
+    all: true
+  });
+  const [selectedTasks, setSelectedTasks] = useState([]);
+
+  // rowSelection object needed for the Table component to handle row selection
+  const rowSelection = {
+    selectedRowKeys: selectedTasks,
+    onChange: selectedRowKeys => {
+      setSelectedTasks(selectedRowKeys);
+    }
+  };
+
+  const { t } = i18nReact.useTranslation();
+
+  const columns = [
+    {
+      title: t('Task ID'),
+      dataIndex: 'task_id',
+      key: 'task_id',
+      render: (text, record) => <a onClick={() => ShowTaskLogsHandler(record.task_id)}>{text}</a>
+    },
+    {
+      title: t('User'),
+      dataIndex: ['result', 'username'],
+      key: 'user'
+    },
+    {
+      title: t('Progress'),
+      dataIndex: ['result', 'progress'],
+      key: 'progress'
+    },
+    {
+      title: t('Task Name'),
+      dataIndex: ['result', 'task_name'],
+      key: 'task_name'
+    },
+    {
+      title: t('Parameters'),
+      dataIndex: 'parameters',
+      key: 'parameters',
+      render: (text, record) => (
+        <div>
+          {record.result?.task_name === 'fileupload' && (
+            <span>{`{file name: ${record.result?.qqfilename}}`}</span>
+          )}
+          {record.result?.task_name === 'document_cleanup' && (
+            <span>{`{keep days: ${record.result?.parameters}}`}</span>
+          )}
+          {record.result?.task_name === 'tmp_cleanup' && (
+            <span>{`{cleanup threshold: ${record.result?.parameters}}`}</span>
+          )}
+        </div>
+      )
+    },
+    {
+      title: t('Status'),
+      dataIndex: 'status',
+      key: 'status',
+      render: status => <Tag color={statusTagColor(status)}>{status.toUpperCase()}</Tag>
+    },
+    {
+      title: t('Started'),
+      dataIndex: ['result', 'task_start'],
+      key: 'started',
+      render: text => formatTimestamp(text)
+    },
+    {
+      title: t('Duration'),
+      key: 'duration',
+      render: (_, record) => calculateDuration(record.result?.task_start, record.date_done)
+    }
+  ];
+
+  // Custom function to determine Tag color based on status
+  const statusTagColor = status => {
+    switch (status.toUpperCase()) {
+      case 'SUCCESS':
+        return 'success';
+      case 'FAILURE':
+        return 'error'; // Ant Design does not have 'failure', using 'error' instead
+      case 'RUNNING':
+        return 'warning';
+      default:
+        return '';
+    }
+  };
+
+  // Function to calculate row class names based on status
+  const getRowClassName = record => {
+    switch (record.status.toUpperCase()) {
+      case 'SUCCESS':
+        return 'row-success';
+      case 'FAILURE':
+        return 'row-failure';
+      case 'RUNNING':
+        return 'row-running';
+      default:
+        return '';
+    }
+  };
+
+  useEffect(() => {
+    const fetchTasks = () => {
+      get('/desktop/api2/taskserver/get_taskserver_tasks/', null, {
+        transformResponse: data => {
+          return data;
+        }
+      })
+        .then(tasks => {
+          if (Array.isArray(tasks)) {
+            const sortedTasks = tasks.sort((a, b) => {
+              const dateA = new Date(a.date_done);
+              const dateB = new Date(b.date_done);
+              if (!isNaN(dateA) && !isNaN(dateB)) {
+                return dateB - dateA;
+              }
+              return 0;
+            });
+            setTasks(sortedTasks);
+          } else {
+            act(() =>
+              huePubSub.publish('hue.global.error', {
+                message: `Expected an array of tasks, but received: ${tasks}`
+              })
+            );
+          }
+        })
+        .catch(error => {
+          const errorMessage = extractErrorMessage(error);
+          act(() =>
+            huePubSub.publish('hue.global.error', {
+              message: `Error fetching tasks from redis ${errorMessage}`
+            })
+          );
+        })
+        .finally(() => {
+          setTimeout(fetchTasks, 5000);
+        });
+    };
+
+    fetchTasks(); //fetch tasks initially
+  }, []);
+
+  const handleSearchChange = e => {
+    setSearchTerm(e.target.value.toLowerCase());
+  };
+
+  const handleStatusFilterChange = status => {
+    setStatusFilter(prevStatusFilter => {
+      const isAll = status === 'all';
+      const newStatusFilter = {
+        ...prevStatusFilter,
+        [status]: isAll ? true : !prevStatusFilter[status]
+      };
+
+      if (isAll) {
+        // If 'all' is selected, set everything else to false
+        newStatusFilter.success = false;
+        newStatusFilter.failure = false;
+        newStatusFilter.running = false;
+      } else {
+        // If any specific status is toggled, set 'all' to false
+        newStatusFilter.all = false;
+      }
+
+      // If no individual statuses are selected, default to 'all'
+      if (!newStatusFilter.success && !newStatusFilter.failure && !newStatusFilter.running) {
+        newStatusFilter.all = true;
+      }
+
+      return newStatusFilter;
+    });
+  };
+
+  enum TaskStatus {
+    Success = 'SUCCESS',
+    Failure = 'FAILURE',
+    Running = 'RUNNING'
+  }
+
+  interface Task_Status {
+    result?: {
+      task_name: string;
+      username: string;
+    };
+    task_id: string;
+    status: TaskStatus;
+  }
+
+  const filteredTasks = tasks.filter(task => {
+    const taskNameMatch = task.result?.task_name?.toLowerCase().includes(searchTerm);
+    const userIdMatch = task.result?.username?.toLowerCase().includes(searchTerm);
+    const taskIdMatch = task.task_id?.toLowerCase().includes(searchTerm);
+    const statusMatch =
+      statusFilter.all ||
+      (statusFilter.success && task.status === TaskStatus.Success) ||
+      (statusFilter.failure && task.status === TaskStatus.Failure) ||
+      (statusFilter.running && task.status === TaskStatus.Running);
+
+    return (taskNameMatch || userIdMatch || taskIdMatch) && statusMatch;
+  });
+
+  const handleKillTask = taskId => {
+    post(`/desktop/api2/taskserver/kill_task/${taskId}/`)
+      .then(response => {
+        // Assuming the server response is in the expected format
+        const { status, message } = response;
+        if (status === 'success') {
+          act(() =>
+            huePubSub.publish('hue.global.error', { message: `Task: ${taskId} has been killed.` })
+          );
+        } else if (status === 'info') {
+          act(() =>
+            huePubSub.publish('', {
+              message: `Task: ${taskId} has already been completed or revoked.`
+            })
+          );
+        } else {
+          act(() =>
+            huePubSub.publish('hue.global.error', {
+              message: `Task: ${taskId} could not be killed. ${message}`
+            })
+          );
+        }
+      })
+      .catch(error => {
+        const errorMessage = extractErrorMessage(error);
+        act(() =>
+          huePubSub.publish('hue.global.error', {
+            message: `Task not killed - Error: ${errorMessage}`
+          })
+        );
+      });
+  };
+
+  const handleKillSelectedTasks = () => {
+    selectedTasks.forEach(taskId => {
+      handleKillTask(taskId);
+    });
+    setSelectedTasks([]);
+  };
+
+  // Render the table with the tasks data
+  return (
+    <div className="content-panel-inner">
+      <div className="flex-container-style">
+        <PrimaryButton onClick={handleSchedulePopup}>Schedule Task</PrimaryButton>
+        <DangerButton onClick={handleKillSelectedTasks}>Kill Task</DangerButton>
+        <Input
+          type="text"
+          placeholder="Search by task name, user ID, or task ID..."
+          onChange={handleSearchChange}
+          className="button-input-style"
+          style={{ flexGrow: 1 }}
+        />
+        <Checkbox
+          checked={statusFilter.success}
+          onChange={() => handleStatusFilterChange('success')}
+        />
+        <span className="success-text">Succeeded</span>
+        <Checkbox
+          checked={statusFilter.running}
+          onChange={() => handleStatusFilterChange('running')}
+        />
+        <span className="running-text">Running</span>
+        <Checkbox
+          checked={statusFilter.failure}
+          onChange={() => handleStatusFilterChange('failure')}
+        />
+        <span className="failed-text">Failed</span>
+        <Checkbox checked={statusFilter.all} onChange={() => handleStatusFilterChange('all')} />
+        All
+      </div>
+      <Table
+        rowSelection={rowSelection}
+        columns={columns}
+        // eslint-disable-next-line @typescript-eslint/no-unused-vars
+        dataSource={filteredTasks.map(({ children, ...task }) => task)}
+        rowKey="task_id"
+        rowClassName={getRowClassName}
+        pagination={false}
+        size="small"
+      />
+    </div>
+  );
+};
+
+export default TaskBrowser;

+ 3 - 0
desktop/core/src/desktop/js/reactComponents/imports.js

@@ -26,6 +26,9 @@ export async function loadComponent(name) {
     case 'WelcomeTour':
       return (await import('./WelcomeTour/WelcomeTour')).default;
 
+    case 'TaskBrowser':
+      return (await import('./TaskBrowser/TaskBrowser')).default;
+
     default:
       console.error(`A placeholder component is rendered because you probably forgot to include your new component in the 
       loadComponent function of reactComponents/imports.js`);

+ 70 - 0
desktop/core/src/desktop/js/utils/dateTimeUtils.ts

@@ -0,0 +1,70 @@
+// Licensed to Cloudera, Inc. under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  Cloudera, Inc. licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+/**
+ * Calculates the duration between two dates.
+ * @param start The start date.
+ * @param end The end date.
+ * @returns The duration as a string in the format "hh:mm:ss".
+ */
+export const calculateDuration = (start: Date | string, end: Date | string): string => {
+  if (!start || !end) {
+    return 'N/A'; // If either start or end is missing
+  }
+
+  const startDate = new Date(start);
+  const endDate = new Date(end);
+  const duration = endDate.getTime() - startDate.getTime(); // Duration in milliseconds
+
+  if (isNaN(duration)) {
+    return 'Invalid Dates'; // If dates can't be parsed
+  }
+
+  // Convert duration to hours, minutes, and seconds
+  const seconds = Math.floor((duration / 1000) % 60);
+  const minutes = Math.floor((duration / (1000 * 60)) % 60);
+  const hours = Math.floor(duration / (1000 * 60 * 60));
+
+  // Build duration string
+  let durationStr = '';
+  if (hours > 0) {
+    durationStr += `${hours}h, `;
+  }
+  if (minutes > 0 || hours > 0) {
+    durationStr += `${minutes}m, `;
+  }
+  durationStr += `${seconds}s`;
+
+  return durationStr;
+};
+
+/**
+ * Formats a timestamp to a readable string.
+ * @param timestamp The timestamp to format.
+ * @returns The formatted timestamp as a string.
+ */
+export const formatTimestamp = (timestamp: Date | string): string => {
+  const options = {
+    year: 'numeric',
+    month: 'long',
+    day: 'numeric',
+    hour: '2-digit',
+    minute: '2-digit',
+    hour12: true
+  };
+  const date = new Date(timestamp);
+  return date.toLocaleString('en-US', options);
+};

+ 3 - 0
desktop/core/src/desktop/templates/about_layout.mako

@@ -75,6 +75,9 @@ def is_selected(section, matcher):
                   <a href="${ url('desktop.lib.metrics.views.index') }">${_('Metrics')}</a>
                 </li>
                 % endif
+                <li class="${is_selected(section, 'task_server')}">
+                  <a href="${ url('desktop.views.task_server_view') }">${_('Task Server')}</a>
+                </li>
               % endif
             </ul>
           </div>

+ 1 - 0
desktop/core/src/desktop/templates/global_js_constants.mako

@@ -104,6 +104,7 @@
     dump_config: { url: '/desktop/dump_config', title: '${_('Dump Configuration')}' },
     threads: { url: '/desktop/debug/threads', title: '${_('Threads')}' },
     metrics: { url: '/desktop/metrics', title: '${_('Metrics')}' },
+    taskserver: { url: '/task_server', title: '${_('Task Server')}' },
     connectors: { url: '/desktop/connectors', title: '${_('Connectors')}' },
     analytics: { url: '/desktop/analytics', title: '${_('Analytics')}' },
     sqoop: { url: '/sqoop', title: '${_('Sqoop')}' },

+ 1 - 0
desktop/core/src/desktop/templates/hue.mako

@@ -249,6 +249,7 @@ ${ hueIcons.symbols() }
         <div id="embeddable_dump_config" class="embeddable"></div>
         <div id="embeddable_threads" class="embeddable"></div>
         <div id="embeddable_metrics" class="embeddable"></div>
+        <div id="embeddable_taskserver" class="embeddable"></div>
         <div id="embeddable_connectors" class="embeddable"></div>
         <div id="embeddable_analytics" class="embeddable"></div>
         <div id="embeddable_403" class="embeddable"></div>

+ 40 - 0
desktop/core/src/desktop/templates/taskserver_list_tasks.mako

@@ -0,0 +1,40 @@
+<%!
+import sys
+from desktop.views import commonheader, commonfooter
+from desktop.auth.backend import is_admin
+
+if sys.version_info[0] > 2:
+  from django.utils.translation import gettext as _
+else:
+  from django.utils.translation import ugettext as _
+%>
+
+<%namespace name="actionbar" file="actionbar.mako" />
+<%namespace name="layout" file="about_layout.mako" />
+
+%if not is_embeddable:
+${ commonheader(_('Task Server'), "about", user, request) | n,unicode }
+%endif
+
+${ layout.menubar(section='task_server') }
+
+
+<div class="container-fluid">
+  <div class="card card-small">
+
+    <script type="text/javascript">
+      (function () {
+        window.createReactComponents('#taskbrowser-container');
+      })();
+    </script>
+
+    <div id="taskbrowser-container">
+      <MyComponent data-reactcomponent='TaskBrowser' data-props='{"myObj": {"id": 1}, "children": "mako template only", "version" : "${sys.version_info[0]}"}' ></MyComponent>
+    </div>
+
+  </div>
+</div>
+
+%if not is_embeddable:
+${ commonfooter(request, messages) | n,unicode }
+%endif