Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: 修复load .wasm文件对importObject的处理 #1705

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/mako/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ percent-encoding = { version = "2.3.1" }
serde = { workspace = true }
serde_json = { workspace = true }
url = { version = "2.5.0" }
wasmparser = "0.207.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

建议更新 wasmparser 依赖版本

当前使用的版本 (0.207.0) 已过时。最新的稳定版本是 0.220.0,建议:

  1. 更新到最新版本
  2. 使用版本范围声明以自动获取补丁更新:wasmparser = "^0.220.0"

安全检查显示该包没有已知的安全漏洞。

🔗 Analysis chain

建议检查 wasmparser 的版本选择

建议考虑以下几点:

  1. 使用版本范围声明(例如 ^0.207.0)来自动接收补丁版本的更新
  2. 确认当前选择的版本是否为最新的稳定版本

运行以下脚本来验证版本信息:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# 检查 wasmparser 包的最新版本和安全公告

# 从 crates.io 获取最新版本信息
curl -s https://crates.io/api/v1/crates/wasmparser | jq -r '.crate.max_stable_version'

# 检查安全公告
gh api graphql -f query='
{
  securityVulnerabilities(first: 5, ecosystem: RUST, package: "wasmparser") {
    nodes {
      advisory {
        summary
        severity
        publishedAt
      }
      vulnerableVersionRange
      firstPatchedVersion {
        identifier
      }
    }
  }
}'

Length of output: 439


swc_core = { workspace = true, features = [
"base",
Expand Down
91 changes: 90 additions & 1 deletion crates/mako/src/plugins/wasm_runtime.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
use std::collections::HashMap;
use std::fs::File;
use std::io::Read;
use std::sync::Arc;

use anyhow;
use wasmparser::{Import, Parser, Payload};

use crate::ast::file::{Content, JsContent};
use crate::compiler::Context;
use crate::plugin::Plugin;
use crate::plugin::{Plugin, PluginLoadParam};

pub struct WasmRuntimePlugin {}

Expand All @@ -27,4 +32,88 @@ impl Plugin for WasmRuntimePlugin {
Ok(vec![])
}
}

fn load(
&self,
param: &PluginLoadParam,
_context: &Arc<Context>,
) -> anyhow::Result<Option<Content>> {
let file = param.file;
if file.path.to_string_lossy().ends_with(".wasm") {
let final_file_name = format!(
"{}.{}.{}",
file.get_file_stem(),
file.get_content_hash()?,
file.extname
);
let origin_path = file.pathname.to_string_lossy().to_string();
_context.emit_assets(origin_path, final_file_name.clone());

let mut buffer = Vec::new();
File::open(&file.path)?.read_to_end(&mut buffer)?;
// Parse wasm file to get imports
let mut import_objs_map: HashMap<&str, Vec<String>> = HashMap::new();
for payload in Parser::new(0).parse_all(&buffer) {
if let Ok(Payload::ImportSection(imports)) = payload {
for import in imports {
if let Ok(Import {
module,
name,
ty: _,
}) = import
{
if let Some(import_obj) = import_objs_map.get_mut(module) {
import_obj.push(name.to_string());
} else {
import_objs_map.insert(module, vec![name.to_string()]);
}
}
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

优化WASM导入解析的错误处理

当前代码在处理导入时的错误处理可以更加完善。建议:

- if let Ok(Import { module, name, ty: _ }) = import
+ match import {
+     Ok(Import { module, name, ty: _ }) => {
+         if let Some(import_obj) = import_objs_map.get_mut(module) {
+             import_obj.push(name.to_string());
+         } else {
+             import_objs_map.insert(module, vec![name.to_string()]);
+         }
+     },
+     Err(e) => log::warn!("解析WASM导入时出错: {}", e)
+ }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let mut import_objs_map: HashMap<&str, Vec<String>> = HashMap::new();
for payload in Parser::new(0).parse_all(&buffer) {
if let Ok(Payload::ImportSection(imports)) = payload {
for import in imports {
if let Ok(Import {
module,
name,
ty: _,
}) = import
{
if let Some(import_obj) = import_objs_map.get_mut(module) {
import_obj.push(name.to_string());
} else {
import_objs_map.insert(module, vec![name.to_string()]);
}
}
}
}
}
let mut import_objs_map: HashMap<&str, Vec<String>> = HashMap::new();
for payload in Parser::new(0).parse_all(&buffer) {
if let Ok(Payload::ImportSection(imports)) = payload {
for import in imports {
match import {
Ok(Import { module, name, ty: _ }) => {
if let Some(import_obj) = import_objs_map.get_mut(module) {
import_obj.push(name.to_string());
} else {
import_objs_map.insert(module, vec![name.to_string()]);
}
},
Err(e) => log::warn!("解析WASM导入时出错: {}", e)
}
}
}
}


let mut js_imports_str = String::new();
let mut import_objs_str = String::new();

for (index, (key, value)) in import_objs_map.iter().enumerate() {
js_imports_str.push_str(&format!(
"import * as module{module_idx} from \"{module}\";\n",
module_idx = index,
module = key
));

import_objs_str.push_str(&format!(
"\"{module}\": {{ {names} }}",
module = key,
names = value
.iter()
.map(|name| format!("\"{}\": module{}[\"{}\"]", name, index, name))
.collect::<Vec<String>>()
.join(", ")
));
}

let mut content = String::new();
content.push_str(&js_imports_str);

if import_objs_str.is_empty() {
content.push_str(&format!(
"module.exports = require._interopreRequireWasm(exports, \"{}\")",
final_file_name
));
} else {
content.push_str(&format!(
"module.exports = require._interopreRequireWasm(exports, \"{}\", {{{}}})",
Comment on lines +106 to +111
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

检查require._interopreRequireWasm函数名是否正确

在生成的JavaScript代码中,使用了require._interopreRequireWasm函数。请确认函数名是否拼写正确,是否应为require._interopRequireWasm

如果存在拼写错误,建议修改如下:

- "module.exports = require._interopreRequireWasm(exports, \"{}\")",
+ "module.exports = require._interopRequireWasm(exports, \"{}\")",

Committable suggestion skipped: line range outside the PR's diff.


💡 Codebase verification

函数名称拼写错误

wasm_runtime.js中,函数名为requireModule._interopreRequireWasm,但在生成的代码中使用了require._interopreRequireWasm。此外,函数名中存在拼写错误:_interopreRequireWasm应为_interopRequireWasminterop而不是interopre)。

  • crates/mako/src/plugins/wasm_runtime.rs:需要修正函数调用名称为requireModule._interopRequireWasm
  • crates/mako/src/plugins/wasm_runtime/wasm_runtime.js:需要修正函数定义名称中的拼写错误
🔗 Analysis chain

验证require._interopRequireWasm函数的定义

请确认require._interopRequireWasm函数在代码库中已正确定义,以确保生成的JavaScript代码能够正常运行。

运行以下脚本以检查该函数的定义:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# 描述:在代码库中搜索`_interopRequireWasm`函数的定义

# 测试:在所有JavaScript文件中查找`_interopRequireWasm`的定义,期望能够找到函数定义的位置。
rg '_interopRequireWasm' --type js

Length of output: 34


Script:

#!/bin/bash
# 扩大搜索范围,不限制文件类型
rg -l "_interopreRequireWasm"

# 搜索可能的runtime或helper文件
fd -e js -e ts "runtime|helper" --type f

# 检查项目根目录下的配置文件,查找相关依赖或配置
fd "package.json|webpack|rollup|vite" --max-depth 2 --type f --exec cat {}

Length of output: 3456


Script:

#!/bin/bash
# 检查wasm_runtime.js文件中的_interopreRequireWasm函数定义
cat crates/mako/src/plugins/wasm_runtime/wasm_runtime.js

# 检查其他可能引用此函数的文件内容
cat crates/mako/src/build/load.rs

Length of output: 12053

final_file_name, import_objs_str
));
}

return Ok(Some(Content::Js(JsContent {
content,
..Default::default()
})));
}

Ok(None)
}
}
5 changes: 5 additions & 0 deletions examples/import-resources/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import toml, { title } from './index.toml';
import xml from './index.xml';
import yaml, { pi } from './index.yaml';
import MailchimpUnsplash from './mailchimp-unsplash.jpg';
import * as wasm from './minus-wasm-pack';

const num1 = 10;
const num2 = 20;
Expand All @@ -32,6 +33,10 @@ function App() {
<h2>
Test import .wasm file async: {num1} + {num2} = {sum}
</h2>
<h2>
Test import .wasm file(generated by wasm-pack) async: {num1} - {num2} ={' '}
{wasm.minus(num1, num2)}
</h2>
<div>
<h2>Test import .toml file</h2>
<pre>{JSON.stringify(toml, null, 2)}</pre>
Expand Down
15 changes: 15 additions & 0 deletions examples/import-resources/minus-wasm-pack/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/* tslint:disable */
/* eslint-disable */
/**
*/
export function greet(): void;
/**
* @param {string} name
*/
export function greet2(name: string): void;
/**
* @param {number} a
* @param {number} b
* @returns {number}
*/
export function minus(a: number, b: number): number;
4 changes: 4 additions & 0 deletions examples/import-resources/minus-wasm-pack/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
import { __wbg_set_wasm } from './index_bg.js';
import * as wasm from './index_bg.wasm';
__wbg_set_wasm(wasm);
export * from './index_bg.js';
133 changes: 133 additions & 0 deletions examples/import-resources/minus-wasm-pack/index_bg.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
let wasm;
export function __wbg_set_wasm(val) {
wasm = val;
}

const lTextDecoder =
typeof TextDecoder === 'undefined'
? (0, module.require)('util').TextDecoder
: TextDecoder;

let cachedTextDecoder = new lTextDecoder('utf-8', {
ignoreBOM: true,
fatal: true,
});

cachedTextDecoder.decode();

let cachedUint8ArrayMemory0 = null;

function getUint8ArrayMemory0() {
if (
cachedUint8ArrayMemory0 === null ||
cachedUint8ArrayMemory0.byteLength === 0
) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
Comment on lines +21 to +28
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

确保内存视图在内存增长时更新

getUint8ArrayMemory0() 中,缓存的 cachedUint8ArrayMemory0 可能在 WebAssembly 内存增长后失效。建议在检测到内存增长时,重新获取内存视图。


function getStringFromWasm0(ptr, len) {
ptr = ptr >>> 0;
return cachedTextDecoder.decode(
getUint8ArrayMemory0().subarray(ptr, ptr + len),
);
}
/**
*/
export function greet() {
wasm.greet();
}

let WASM_VECTOR_LEN = 0;

const lTextEncoder =
typeof TextEncoder === 'undefined'
? (0, module.require)('util').TextEncoder
: TextEncoder;

let cachedTextEncoder = new lTextEncoder('utf-8');

const encodeString =
typeof cachedTextEncoder.encodeInto === 'function'
? function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
}
: function (arg, view) {
const buf = cachedTextEncoder.encode(arg);
view.set(buf);
return {
read: arg.length,
written: buf.length,
};
};

function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length, 1) >>> 0;
getUint8ArrayMemory0()
.subarray(ptr, ptr + buf.length)
.set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}

let len = arg.length;
let ptr = malloc(len, 1) >>> 0;

const mem = getUint8ArrayMemory0();

let offset = 0;

for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7f) break;
mem[ptr + offset] = code;
}

if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, (len = offset + arg.length * 3), 1) >>> 0;
const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);

offset += ret.written;
ptr = realloc(ptr, len, offset, 1) >>> 0;
Comment on lines +94 to +98
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

更新内存指针以防止访问错误

在内存重分配后,ptr 被更新,但 memview 仍指向旧的内存区域。应在重分配后重新获取内存视图和缓冲区,以避免可能的内存访问错误。

}

WASM_VECTOR_LEN = offset;
return ptr;
}
Comment on lines +65 to +103
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

优化字符串传递到 WASM 的实现

passStringToWasm0() 函数较为复杂,可能存在对多字节字符处理的不稳定性。建议使用 TextEncoder 的 encode() 方法简化字符串编码过程,并重新评估内存分配和重分配逻辑。

/**
* @param {string} name
*/
export function greet2(name) {
const ptr0 = passStringToWasm0(
name,
wasm.__wbindgen_malloc,
wasm.__wbindgen_realloc,
);
const len0 = WASM_VECTOR_LEN;
wasm.greet2(ptr0, len0);
}

/**
* @param {number} a
* @param {number} b
* @returns {number}
*/
export function minus(a, b) {
const ret = wasm.minus(a, b);
return ret;
}

export function __wbg_alert_f837f172b2a24942(arg0, arg1) {
alert(getStringFromWasm0(arg0, arg1));
}

export function __wbg_prompt_ec584a06a1c7c28b(arg0, arg1) {
prompt(getStringFromWasm0(arg0, arg1));
}
Binary file not shown.
13 changes: 13 additions & 0 deletions examples/import-resources/minus-wasm-pack/index_bg.wasm.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/* tslint:disable */
/* eslint-disable */
export const memory: WebAssembly.Memory;
export function greet2(a: number, b: number): void;
export function minus(a: number, b: number): number;
export function greet(): void;
export function __wbindgen_malloc(a: number, b: number): number;
export function __wbindgen_realloc(
a: number,
b: number,
c: number,
d: number,
): number;