Skip to content

Commit

Permalink
Updated remote procedure examples; Refactored remote procedure blueprint
Browse files Browse the repository at this point in the history
  • Loading branch information
meta-paul committed Nov 5, 2024
1 parent 9e8bb15 commit f48d6cf
Show file tree
Hide file tree
Showing 57 changed files with 852 additions and 428 deletions.
5 changes: 4 additions & 1 deletion docs/web/docs/guides/tutorials/model_in_the_loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,10 @@ docker-compose -f docker/docker-compose.dev.yml run \
python /mephisto/examples/remote_procedure/interactive_image_generation/run_task__local__inhouse.py
```

3. Once your Task launches, your console will display you URLs like this: `http://localhost:3001?worker_id=WORKER_USERNAME&id=1`
3. Once your Task launches, your console will display you Task Unit URLs like this: `http://localhost:3001?worker_id=WORKER_USERNAME&id=1`

4. Opening a Task Unit URL will show you a page like this:
![Model-in-the-loop](./screenshots/model_in_the_loop.jpg)


#### Further Details
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ function ReviewApp() {
}
window.addEventListener("resize", updateSize);
updateSize();
// HACK: Catch-all resize, if normal resizes failed (e.g. acync long loading images)
// HACK: Catch-all resize, if normal resizes failed (e.g. slow acync loading of images)
setTimeout(() => {
updateSize();
}, 3000);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ function ReviewApp() {
}
window.addEventListener("resize", updateSize);
updateSize();
// HACK: Catch-all resize, if normal resizes failed (e.g. acync long loading images)
// HACK: Catch-all resize, if normal resizes failed (e.g. slow acync loading of images)
setTimeout(() => {
updateSize();
}, 3000);
Expand Down
36 changes: 36 additions & 0 deletions examples/remote_procedure/elementary_remote_procedure/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
<!---
Copyright (c) Meta Platforms and its affiliates.
This source code is licensed under the MIT license found in the
LICENSE file in the root directory of this source tree.
-->

# Mephisto Elementary remote procedure example

This task is a _very simplistic_ bare-bones example of being able to set up a task where the frontend directly connects to the backend
using the `useMephistoRemoteProcedureTask` hook from the `mephisto-task` package.
It should serve as a decent example of how to get a `RemoteProcedureBlueprint` task up off the ground.

Deploying the task as-is brings up a page where the user needs to click the "query backend" button enough times
for the task to be considered ready to submit.

It makes use of the function_registry to use a `"handle_with_model"` method that, admittedly, doesn't use any models,
but hopefully demonstrates where a model can fit into this type of task.

As it stands, these queries are still just a single request to single response model,
but that should be sufficient for most tasks that want to have a backend in-the-loop for an otherwise frontend-heavy task.

## End-to-end example

Now let's see how the whole end-to-end list of commands looks like for the example `Elementary remote procedure`:

```shell
# 1. In your console

docker-compose -f docker/docker-compose.dev.yml up
docker exec -it mephisto_dc bash

# 2.Inside Docker container

cd /mephisto/examples/remote_procedure/elementary_remote_procedure
python ./run_task__local__inhouse.py
```
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,28 @@
defaults:
- /mephisto/blueprint: remote_procedure
- /mephisto/architect: local
- /mephisto/provider: mock
- /mephisto/provider: inhouse
mephisto:
blueprint:
task_source: ${task_dir}/webapp/build/bundle.js
task_source: ${task_dir}/webapp/build/bundle.remote_procedure.js
task_source_review: ${task_dir}/webapp/build/bundle.review.js
link_task_source: false
# NOTE pick something based on your task
block_qualification: test_qual_block
units_per_assignment: 1
log_level: "debug"
provider:
ui_base_url: "http://localhost:3001"
task:
allowed_concurrent: 1
task_name: remote-procedure-test-task
task_name: "Elementary remote procedure"
task_title: "Test a static task that also has access to live backend queries"
# NOTE you'll want to update your task description
task_description: "You will be shown a form that you can submit at any time, but can also toggle a backend query"
# NOTE set a reasonable reward
task_reward: 0.05
# NOTE will want real tags
task_tags: "test,task,fix-me"
task_tags: "test,task,fix-me,elementary_remote_procedure"
# NOTE Model-in-the-loop tasks need to be careful to configure only as many concurrent
# connections as their model can handle at once
max_num_concurrent_units: 40
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def _onboarding_always_valid(onboarding_data: dict) -> bool:
return True


@task_script(default_config_file="example_local_mock")
@task_script(default_config_file="example__local__inhouse")
def main(operator: Operator, cfg: DictConfig) -> None:
examples.build_remote_procedure_template(
force_rebuild=cfg.mephisto.task.force_rebuild,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* LICENSE file in the root directory of this source tree.
*/

describe("Loads remote_procedure_template", () => {
describe("Loads elementary_remote_procedure", () => {
it("Makes request for agent", () => {
cy.intercept({ pathname: "/request_agent" }).as("agentRequest");
cy.visit("/");
Expand Down
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
{
"name": "remote-procedure-template-example",
"name": "elementary-remote-procedure",
"version": "1.0.1",
"description": "",
"main": "webpack.config.js",
"scripts": {
"dev": "webpack --mode development",
"test": "cypress open"
"test": "cypress open",
"build:remote_procedure": "webpack --config=webpack.config.remote_procedure.js --mode development",
"build:review": "webpack --config=webpack.config.review.js --mode development"
},
"keywords": [],
"author": "",
"dependencies": {
"bootstrap": "^4.6.0",
"mephisto-task": "2.0.4",
"bootstrap": "^5.3.1",
"rc-slider": "^8.6.3",
"react": "^16",
"react-bootstrap": "^1.6.0",
"react-dom": "^16",
"react-table": "^6.8.6"
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,17 @@
* LICENSE file in the root directory of this source tree.
*/

import React from "react";
import ReactDOM from "react-dom";
import { BaseFrontend, LoadingScreen } from "./components/core_components.jsx";

import {
ErrorBoundary,
MephistoContext,
useMephistoRemoteProcedureTask,
ErrorBoundary,
} from "mephisto-core";
import React from "react";
import ReactDOM from "react-dom";
import {
LoadingScreen,
ElementaryRemoteProcedureTaskFrontend,
} from "./components/core_components_remote_procedure.jsx";

/* ================= Application Components ================= */

Expand Down Expand Up @@ -51,27 +53,32 @@ function RemoteProcedureApp() {
// At the moment, this task has no onboarding
return <h1>This task doesn't currently have an onboarding example set</h1>;
}

if (blockedReason !== null) {
return <h1>{blockedExplanation}</h1>;
}

if (isLoading) {
return <LoadingScreen />;
}

if (isPreview) {
if (!taskConfig.has_preview) {
return <TaskPreviewView description={taskConfig.task_description} />;
}

if (previewHtml === null) {
return <div>Loading...</div>;
}

return <div dangerouslySetInnerHTML={{ __html: previewHtml }} />;
}

return (
<ErrorBoundary handleError={handleFatalError}>
<MephistoContext.Provider value={mephistoProps}>
<div className="container-fluid" id="ui-container">
<BaseFrontend
<div className={"container"} id={"ui-container"}>
<ElementaryRemoteProcedureTaskFrontend
taskData={initialTaskData}
handleRemoteCall={handleRemoteCall}
handleSubmit={handleSubmit}
Expand All @@ -84,7 +91,7 @@ function RemoteProcedureApp() {

function TaskPreviewView({ description }) {
return (
<div className="preview-screen">
<div className={"preview-screen"}>
<div
dangerouslySetInnerHTML={{
__html: description,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
/*
* Copyright (c) Meta Platforms and its affiliates.
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

import React from "react";

function LoadingScreen() {
return <Directions>Loading...</Directions>;
}

function Directions({ children }) {
return (
<div className={"card mb-4"}>
<div className={"card-body container"}>{children}</div>
</div>
);
}

function Instructions() {
return (
<div className={"mb-5"}>
<h1 data-cy={"directions-header"}>
Elementary example of remote procedure
</h1>

<h5 data-cy={"subheader-paragraph"}>
This is a simple task to demonstrate communication with the server
</h5>

<p data-cy={"directions-paragraph"}>
To submit this task, you must make a few backend queries first
</p>
</div>
);
}

function ElementaryRemoteProcedureTaskFrontend({
taskData,
handleRemoteCall,
handleSubmit,
finalResults = null,
}) {
const inReviewState = finalResults !== null;

if (!inReviewState && !taskData) {
return <LoadingScreen />;
}

const [queryCount, setQueryCount] = React.useState(0);
let canSubmit = queryCount > 3;

const disabledQueryButton = inReviewState;
const disabledSubmitButton = inReviewState || !canSubmit;

return (
<div>
{/* Final result of template example (needed only for TaskReview app) */}
{inReviewState && (
<div className={"alert alert-success mb-5 col-6"}>
Task result: {finalResults.backendActionsDone}
</div>
)}

<Instructions />

<button
className={"btn btn-primary me-3"}
onClick={() => {
setQueryCount(queryCount + 1);
handleRemoteCall({
arg1: "hello",
arg2: "goodbye",
arg3: queryCount,
}).then((response) => alert(JSON.stringify(response)));
}}
disabled={disabledQueryButton}
data-cy={"query-backend-button"}
>
Query Backend
</button>

<button
className={"btn btn-success"}
onClick={() =>
handleSubmit({
backendActionsDone: queryCount,
})
}
disabled={disabledSubmitButton}
data-cy={"submit-button"}
>
Submit Task
</button>
</div>
);
}

export { LoadingScreen, ElementaryRemoteProcedureTaskFrontend };
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
/*
* Copyright (c) Meta Platforms and its affiliates.
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

import "bootstrap/dist/css/bootstrap.min.css";
import "./app_remote_procedure.jsx";
import "./css/style.css";
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
import "./app.jsx";
import "./css/style.css";

import "bootstrap/dist/css/bootstrap.min.css";
import "./css/style.css";
import "./reviewapp.jsx";
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
/*
* Copyright (c) 2017-present, Facebook, Inc.
* All rights reserved.
* This source code is licensed under the BSD-style license found in the
* LICENSE file in the root directory of this source tree. An additional grant
* of patent rights can be found in the PATENTS file in the same directory.
*/

import React from "react";
import ReactDOM from "react-dom";
import { ElementaryRemoteProcedureTaskFrontend } from "./components/core_components_remote_procedure.jsx";

function ReviewApp() {
const appRef = React.useRef(null);
const [reviewData, setReviewData] = React.useState(null);

// Requirement #1. Render review components after receiving Task data via message
window.onmessage = function (e) {
const data = JSON.parse(e.data);
setReviewData(data["REVIEW_DATA"]);
};

// Requirement #2. Resize iframe height to fit its content
React.useLayoutEffect(() => {
function updateSize() {
if (appRef.current) {
window.top.postMessage(
JSON.stringify({
IFRAME_DATA: {
height: appRef.current.offsetHeight,
},
}),
"*"
);
}
}
window.addEventListener("resize", updateSize);
updateSize();
// HACK: Catch-all resize, if normal resizes failed (e.g. slow acync loading of images)
setTimeout(() => {
updateSize();
}, 3000);
return () => window.removeEventListener("resize", updateSize);
}, []);

// Requirement #3. This component must return a div with `ref={appRef}`
// so we can get displayed height of this component (for iframe resizing)
return (
<div ref={appRef}>
{reviewData ? (
<div className={"container"} id={"ui-container"}>
<ElementaryRemoteProcedureTaskFrontend
finalResults={reviewData["outputs"]}
/>
</div>
) : (
<div>Loading...</div>
)}
</div>
);
}

ReactDOM.render(<ReviewApp />, document.getElementById("app"));
Loading

0 comments on commit f48d6cf

Please sign in to comment.