TRIPOTHON S1 / VR · XR · AR

上街去 / Carry the Cosmos

行走、种植、相遇,让真实城市成为可以参与的空间。
Walk, plant, and encounter—take part in the real city.

APP + AR · 2:00 · 1920 × 1080 · 原声 · 英文字幕 / Original audio · English subtitles

本项目及本次提交内容均在本届 Tripothon 比赛期间完成。
This project and all materials in this submission were created during this Tripothon.

AR EXPERIENCE / 身体与空间的交互BODY · PAUSE · PRESENCE

Aha Moment of a Flâneur — 漫游者的顿悟时刻

COVER / 封面

在移动的间隙,
重新感到自己与世界的联系。In the pauses within movement,
notice the connection between yourself and the world.

开始体验 Start the encounter

围绕同一个停留时刻的两种相遇,已整合进同一相机入口。Two encounters around the same moment of pause, integrated into one camera entry.

START HERE / 从这里开始

「上街去」AR 体验说明书AR guide for Carry the Cosmos

QUICK START / 开始体验
  1. 01 · 直接进入

    点击左侧 tab「AR 种植」,直接进入相机。

    Tap the AR Plant tab on the left to open the camera.

  2. 02 · 换植物(可选)

    可直接使用当前植物;需要更换时,在相机内点「植物 · 名称」,选一株,再点「选择这株」。

    Use the current plant directly. To change it, tap Plants · name in the camera, pick a species, then tap Choose this plant.

  3. 03 · 种下

    对准地面慢慢移动;落点固定后,点「种下」。

    Scan the ground slowly. Once the spot is fixed, tap Grow.

  4. 04 · 观察

    看它生长、绕着走;点「换个位置」再种一株。

    Watch it grow, walk around it, or tap New spot to plant again.

  5. 05 · 遇鸟

    切到「伸手遇鸟」,摊掌或伸单指停稳;握拳也陪你,移开手送别。

    Switch to Birds. Offer a steady palm or finger. A fist keeps the bird with you; move your hand away to say goodbye.

  6. 06 · 离开与回访

    「返回」退出;准备出门 → 我的植物 → 回访 AR 现场。同一手机回到原处,缓慢扫描周围环境。

    Tap Back to exit. Get ready → My plants → Revisit AR scene. Return with the same phone and slowly scan the original surroundings.

在支持 AR 的 iPhone 上打开「上街去」,首次进入允许相机访问。Open Carry the Cosmos on an AR-capable iPhone and allow camera access.

打开 iPhone 版 Open in TestFlight ↗

左侧入口:最新本地构建截图;种植与遇鸟:iPhone 实景画面。Left entry tab: a screenshot of the latest local build. Planting and bird: real iPhone captures.

FIVE MOMENTS · FIFTEEN PLATES / 五张影像 · 十五张图解

从一个时刻,走进交互的细节。From a moment to the details of its interaction.

THE IDEA

漫游者的顿悟时刻Aha Moment of a Flâneur

把一次普通的停留,作为重新感知身体、环境与自身存在的入口。一个小动作得到回应,一个地点开始与自己有关。

A pause becomes a way to notice the body, the surroundings, and one's own presence. A small gesture receives a response; an ordinary place becomes personally meaningful.

HUMAN MOVEMENT

移动中的间隙The pauses within movement

散步、等待、走累之后停下来。体验允许人观察、尝试、保持不动,也随时重新出发。

During a walk, while waiting, or after stopping to rest, a person can observe, try a gesture, remain still, or simply move on.

PERCEPTION / 感知

THE INTERACTION LOOP

身体如何得到回应How the body receives a response

视觉模型感知手部关键点,ARKit 感知空间;规则引擎处理时序与状态,再由动作、空间变化和轻触觉呈现回应。

A vision model detects hand keypoints, while ARKit senses space. Rules manage timing and state; animation, spatial change, and light haptics express the response.

ENCOUNTER 01 / 掌心宇宙

ENCOUNTER 01 · BIRD

掌心的一次相遇An encounter in the palm

这些画面来自真实手机录屏:伸手、停栖、振翅、再次安定。掌心和指尖,成为小鸟可以回应的落点。

These frames come from actual phone recordings: a hand offered, a bird perched, a wingbeat, and stillness again. The palm and fingertips become responsive perching points.

TIMING & CONTINUITY

让等待成为一种状态Let waiting be a state

短暂遮挡时先等待,确认落点后再靠近;稳定时停栖,支点变化时转移。状态记忆让相遇容纳识别的不确定性。

Wait through a brief occlusion, approach a confirmed perch, rest when the hand is steady, and move when the support changes. State memory accommodates uncertainty in recognition.

BIRD · INTERFACE

界面退后,身体成为输入The body becomes the input

人的手势决定相遇如何继续。鸟的八方向动作素材由引擎编排,呈现停栖、张望与飞行;它是屏幕空间中的 2.5D 数字生命。

The person's gestures shape how the encounter continues. The engine composes eight-direction animation assets into perching, glancing, and flight: a 2.5D digital creature in screen space.

ENCOUNTER 02 / 种下一次停留

ENCOUNTER 02 · PLANT

种下一次停留Plant a moment of stillness

选择眼前的地面,观察一株数字植物展开。人可以留在原处,也可以换一个角度,让自己的移动成为观察的一部分。

Choose a spot on the ground and watch a digital plant unfold. Stay where you are or change your viewpoint, making movement part of the act of observation.

PLANT · AR TECHNOLOGY

AR 技术,如何支撑一次停留AR technology behind a moment of pause

Capacitor 将产品入口连接到 Swift / UIKit 原生界面;ARKit 提供平面检测、射线检测与空间追踪;状态机确认稳定落点、处理追踪中断和恢复;SceneKit 加载 GLB / glTF 2.0 植物模型,呈现约四秒的生长时间线,UIKit 提供轻触觉。交互沿五个环节展开:人的行动 → 产品感知 → 理解与判断 → 产品回应 → 人的反馈。这里的“理解”指规则对空间与时序状态的判断。新版同时保存每株植物记录与本机 ARWorldMap;回到原环境重新定位后,可恢复已保存的落点。

Capacitor connects the app entry to a native Swift / UIKit interface. ARKit provides plane detection, raycasts and spatial tracking; a state machine confirms stable placement and handles tracking loss and recovery. SceneKit loads GLB / glTF 2.0 plant models and renders their roughly four-second growth timeline, while UIKit provides soft haptics. The interaction follows five stages: Human Action → Product Sensing → Understanding / Logic → Product Response → Human Feedback. Here, understanding means rule-based decisions about spatial and temporal state. The latest build also saves individual plant records and an on-device ARWorldMap; saved placements can be restored after relocalizing in the original surroundings.

PLANT · AR INTERFACE

把操作收起,让现场留下来Let the scene take the foreground

实景截图呈现 AR 种植与伸手遇鸟的并列入口。图中标注围绕进入、选择与结束展开,让植物和现场占据主要视野。

The in-place screenshots show AR planting and the bird encounter as adjacent entry points. Annotations explain entering, choosing, and ending, while the plant and its surroundings occupy the main view.

PLANT · BODY & VIEWPOINT

绕着一株植物,重新看见一个地点A new viewpoint on an ordinary place

同一街角郁金香的俯视与侧视实景截图,呈现人改变视角后的观察差异。植物留在现场,人的身体与目光继续移动。

Top and side views of tulips at the same street corner show how observation changes with the viewpoint. The plant remains in the setting while the person's body and attention continue to move.

PLANT · THE COMPLETE FLOW

从选择到生长,再把目光还给街道Choose, grow, observe, and continue

选择植物、确认地面与稳定落点、点按种植、观察生长,再继续行走。实景照片展示种植完成状态;生长节奏与追踪恢复流程依据已核实的实现。

Choose a plant, confirm the ground and a stable placement, tap to plant, watch it grow, and continue walking. The photographs show completed plantings; the growth timing and tracking-recovery flow are based on the verified implementation.

PLANT · EVERYDAY RHYTHM

行走积累,选择慢慢生长Walking accumulates. Choice grows.

植物库共 48 种,初始开放 6 种。此后,每累计一万有效步数,获得一次解锁机会,自由选择一种加入植物库。解锁消耗一次机会,不扣累计步数;步数跨天积累,已解锁植物可反复种植。

The library contains 48 plants, with 6 available initially. Each additional 10,000 valid steps earns one unlock opportunity: choose another plant for the collection. Unlocking uses one opportunity without deducting cumulative steps. Steps carry across days, and unlocked plants can be planted repeatedly.

PLACE & RETURN

让地点成为私人记忆A place becomes a personal memory

新版已保存每株植物记录,可查看地图位置,并用同一手机回到原现场进行 AR 重定位。另有世界模型从地点照片生成三维环境的能力;把同一株植物融合进这种生成场景仍在规划。

The latest build saves individual plant records, shows their map locations, and supports AR relocalization when returning to the original site with the same phone. Separately, a world model generates 3D environments from place photos; integrating the same plant into those generated scenes remains planned.

CALM INTERACTION

回应的分寸The measure of a response

保持、迟疑、离开,都有自己的节奏。设计为等待与结束留出空间,让反馈可以被理解,也可以轻轻放下。

Holding, hesitating, and leaving each have their own rhythm. The design makes room for waiting and ending, so a response can be understood and gently let go.

DESIGN TEST · REFLECTION

一个 Moment,四个交付视角One moment, four design perspectives

从场景与产品思考,到交互流程、视觉动态与可体验原型,围绕同一个停留时刻展开。种植与遇鸟已整合进「上街去」的同一相机,植物记录、地图位置与同机 AR 现场复访已实现;将植物融合进照片生成的三维场景仍在规划。

Scenario and product thinking, interaction flow, visual motion, and an experiential prototype all examine the same pause. Planting and the bird encounter share one camera in Carry the Cosmos. Plant records, map locations, and same-device AR revisits are implemented; integrating plants into photo-generated 3D scenes remains planned.

THE PAUSE / 空白与间隙

CARRY THE COSMOS / 完整产品图册 · 41 页

从一次停留,继续走进城市。

From a moment of pause, back into the city.

伸手与种植,发生在城市漫游中的一次停留。接下来的 41 页继续展开同一个「上街去」:真实地图、城市伙伴、探索与记录。所有页面完整展示。The bird encounter and planting take place during a pause in a city walk. The following 41 pages explore the rest of Carry the Cosmos: real maps, city companions, discovery and records. The complete book follows below.

41 页 · 6 个章节 / 41 pages · 6 chapters

01

产品的出发点Where the idea begins · 1–2

1. 上街去 · 把知识带回现场 / Carry the Cosmos: bring knowledge to the streets
01 / 41
上街去 · 把知识带回现场 Carry the Cosmos: bring knowledge to the streets
2. 三层事件账本 / Three layers, one event ledger
02 / 41
三层事件账本 Three layers, one event ledger
02

把日常走成故事Turn daily walks into stories · 3–14

3. 真实 GPS 同行 / Walking together with real GPS
03 / 41
真实 GPS 同行 Walking together with real GPS
4. 移动、停驻与互动 / Movement, pauses and interaction
04 / 41
移动、停驻与互动 Movement, pauses and interaction
5. 圆形硬件实机 / The round hardware prototype
05 / 41
圆形硬件实机 The round hardware prototype
6. 古籍进入城市 / Bringing ancient books into the city
06 / 41
古籍进入城市 Bringing ancient books into the city
7. 私人知识地图 / A personal knowledge map
07 / 41
私人知识地图 A personal knowledge map
8. Skills Plaza 协议 / The Skills Plaza protocol
08 / 41
Skills Plaza 协议 The Skills Plaza protocol
9. 跨 Skill 旅行规划 / Planning a journey across skills
09 / 41
跨 Skill 旅行规划 Planning a journey across skills
10. 出门准备仪式 / Getting ready to go out
10 / 41
出门准备仪式 Getting ready to go out
11. 伙伴驻留街头 / Companions on the street
11 / 41
伙伴驻留街头 Companions on the street
12. 用步数种下植物 / Planting with your steps
12 / 41
用步数种下植物 Planting with your steps
13. 一路做成一城一刊 / A walk becomes a city journal
13 / 41
一路做成一城一刊 A walk becomes a city journal
14. 相遇、邀请与确认 / Encounters, invitations and confirmation
14 / 41
相遇、邀请与确认 Encounters, invitations and confirmation
03

从屏幕,到口袋From the screen to your pocket · 15–23

15. Forge:生成城市伙伴 / Forge: creating city companions
15 / 41
Forge:生成城市伙伴 Forge: creating city companions
16. 固定伙伴身份包 / A stable companion identity package
16 / 41
固定伙伴身份包 A stable companion identity package
17. Pocket Buddy 成长卡牌 / Pocket Buddy growth cards
17 / 41
Pocket Buddy 成长卡牌 Pocket Buddy growth cards
18. 智能体日报 / Agent Daily
18 / 41
智能体日报 Agent Daily
19. 手机与圆形终端的伙伴显示 / Buddy displays on phone and round device
19 / 41
手机与圆形终端的伙伴显示 Buddy displays on phone and round device
20. 圆形终端正面 / The front of the round device
20 / 41
圆形终端正面 The front of the round device
21. 硬件接口与背面 / Hardware ports and the rear shell
21 / 41
硬件接口与背面 Hardware ports and the rear shell
22. 软硬件共享身份 / A shared identity across software and hardware
22 / 41
软硬件共享身份 A shared identity across software and hardware
23. 硬件五层链路 / The five-layer hardware pipeline
23 / 41
硬件五层链路 The five-layer hardware pipeline
04

让能力进入城市Bring skills into the city · 24–26

24. Skill 四步进入城市 / Four steps for a skill to enter the city
24 / 41
Skill 四步进入城市 Four steps for a skill to enter the city
25. Qwen 基座与能力包 / The Qwen base model and skill packages
25 / 41
Qwen 基座与能力包 The Qwen base model and skill packages
26. Skill 协议执行链路 / Executing the skill protocol
26 / 41
Skill 协议执行链路 Executing the skill protocol
05

让知识回到现场Bring knowledge back to its place · 27–37

27. 古籍 LoRA 固定测试 / Fixed-set testing for the ancient-book LoRA
27 / 41
古籍 LoRA 固定测试 Fixed-set testing for the ancient-book LoRA
28. 碑拓 LoRA 固定测试 / Fixed-set testing for the rubbing LoRA
28 / 41
碑拓 LoRA 固定测试 Fixed-set testing for the rubbing LoRA
29. 碑拓修复与落位闭环 / Restoring rubbings and mapping them to places
29 / 41
碑拓修复与落位闭环 Restoring rubbings and mapping them to places
30. 数字化补全 / Digital completion
30 / 41
数字化补全 Digital completion
31. 多视角 2.5D 展品 / Multi-view 2.5D exhibits
31 / 41
多视角 2.5D 展品 Multi-view 2.5D exhibits
32. 展品落位知识地图 / Placing exhibits on the knowledge map
32 / 41
展品落位知识地图 Placing exhibits on the knowledge map
33. 古籍活化:完整流程 / Reviving ancient books: the complete flow
33 / 41
古籍活化:完整流程 Reviving ancient books: the complete flow
34. 古籍复原、释读与落位 / Restoration, reading and mapping of ancient books
34 / 41
古籍复原、释读与落位 Restoration, reading and mapping of ancient books
35. 碑拓 LoRA 修复与落位 / Rubbing restoration and mapping with LoRA
35 / 41
碑拓 LoRA 修复与落位 Rubbing restoration and mapping with LoRA
36. 展品、铭文与知识地图 / Exhibits, inscriptions and the knowledge map
36 / 41
展品、铭文与知识地图 Exhibits, inscriptions and the knowledge map
37. 从私人到公共 / From private knowledge to public space
37 / 41
从私人到公共 From private knowledge to public space
06

带到真实街头Out on real streets · 38–41

38. 城市智能体游戏 / A city-based agent game
38 / 41
城市智能体游戏 A city-based agent game
39. 手机软件走入西湖 / The mobile app at West Lake
39 / 41
手机软件走入西湖 The mobile app at West Lake
40. 圆形硬件昼夜同行 / Round hardware from day to night
40 / 41
圆形硬件昼夜同行 Round hardware from day to night
41. 城市卡牌与植物回看 / Revisiting city cards and plants
41 / 41
城市卡牌与植物回看 Revisiting city cards and plants

IN THE AUTHOR'S WORDS

完整阐释The full statement

保留定稿口播的中文原文与英文译文。最新操作与实现进展见前面的体验说明和图解。The final narration and its English translation. Current controls and implementation updates appear in the guide and plates above.

展开全文 Read the full statement

我选择的 Human Movement 场景,是一个人移动过程中的间隙。

The Human Movement scenario I have chosen is a pause within a person's movement.

它可能发生在散步途中、等待的时候,或走累之后的一次停留。人暂时没有明确目标,也没有什么必须完成的事情,只是在那里,发呆、观察,或者带着尚未整理好的情绪。

It might happen during a walk, while waiting, or when stopping to rest after growing tired. For a moment, there is no clear destination and nothing that must be done. A person is simply there: daydreaming, observing, or carrying feelings they have not yet made sense of.

这些空白的间隙里,也许脆弱,也许内心不稳定,也许百无聊赖,但非常迷人。它的迷人,在于人性在这些时刻产生着微妙的变化和移动。

In these empty intervals, we may feel vulnerable, unsettled, or utterly bored. Yet there is something deeply compelling about them: the subtle shifts and movements in our humanity during these moments.

这些时刻难以被描述,只能被感知,言语会让感知生锈。

These moments are difficult to describe. They can only be sensed. Words can make perception rust.

我关注的是:在这样的片刻里,人如何重新感到自己的身体、身边的环境,以及自己与世界之间的联系。产品如何轻轻参与其中,为尚未成形的感受保留空间。

What interests me is how, in such moments, a person might become aware again of their body, their surroundings, and their connection to the world. How might a product participate gently, leaving room for feelings that have yet to take shape?

在这个项目中,我将 Flâneur 理解为一种允许偶遇的漫游状态。人可以偏离路线,可以停下来,也可以暂时没有目的。而 Aha Moment,是一次很轻的觉察:

In this project, I understand the Flâneur as a way of wandering that leaves room for chance encounters. A person can stray from a route, stop, or have no destination for a while. An Aha Moment is a quiet realization:

一个小动作,可以得到回应;一个个普通的地方,也可以与自身产生关联。

A small gesture can receive a response. An ordinary place can become personally meaningful.

我希望把这样的体验整合进「上街去」。需要解决的设计问题是:在短暂停留中,如何让人通过身体动作参与互动、获得回应,同时保留观察、等待和随时结束的选择。

I want to bring these experiences into Carry the Cosmos (上街去). The design question is this: during a brief pause, how can people interact through bodily movement and receive a response, while retaining the choice to observe, wait, or end the interaction at any time?

手机感知身体动作与周围空间,结合情境与时序提供反馈。

The phone senses bodily movement and the surrounding space, then uses context and timing to provide feedback.

一种相遇,是让一只数字小鸟落在掌心。

One encounter begins with a digital bird landing in a person's palm.

人伸出手并停稳,小鸟靠近、落下,随手指与支点的变化调整动作。人可以与它玩一会儿,也可以保持不动,观察它停栖。等待也是互动的一部分。

A person offers a hand and holds it steady. The bird approaches and lands, adjusting its movements as the fingers and available perching points change. The person can play with it for a while, or remain still and watch it perch. Waiting is also part of the interaction.

这段体验由视觉感知、规则引擎与动作表现共同构成。手机端通过 Vision 识别手部关键点;规则引擎结合支点位置、动作持续时间与追踪状态,判断小鸟何时靠近、落下、换位或离开;已有的多方向动作素材呈现这些行为。

This experience brings together visual sensing, a rule engine, and animation. On the phone, Vision detects hand keypoints. The rule engine considers perch position, action duration, and tracking status to determine when the bird approaches, lands, changes position, or leaves. Existing animations in multiple directions bring these behaviors to life.

我尤其关注时间与识别误差的处理。短暂遮挡时,小鸟可以悬停等待;改变手指时,它通过飞行与落下转移支点;手保持稳定时,它也能停栖,偶尔转头张望。引擎保留当前互动状态,让连续的互动能够容纳短暂的识别误差。

I pay particular attention to timing and recognition errors. During a brief occlusion, the bird can hover and wait. When the fingers change position, it flies and lands to move between perching points. When the hand remains steady, it can stay perched, occasionally turning its head to look around. The engine retains the current interaction state, allowing a continuous experience to accommodate brief recognition errors.

另一种相遇,是在眼前种下一株数字植物。

Another encounter begins by planting a digital plant in the space before you.

人选择地面,种下植物,观察枝干、叶片与花朵逐渐展开。原本经过的地点,因为这次种植与观察,与人的停留建立联系。

A person chooses a spot on the ground, plants something, and watches its stems, leaves, and flowers gradually unfold. Through this act of planting and observation, a place they might otherwise pass through becomes connected to their pause.

ARKit 提供地面感知与空间追踪。系统确认稳定落点后,通过规则控制植物的生长节奏,并在追踪中断时暂停。植物固定在眼前的空间里,人可以改变角度、绕着观察,让身体的移动继续参与互动。

ARKit provides ground-plane detection and spatial tracking. Once the system confirms a stable placement, rules control the pace of growth and pause it when tracking is interrupted. The plant stays anchored in the space in front of the person. They can change their viewing angle or walk around it, allowing bodily movement to remain part of the interaction.

植物的选择与日常行走关联。「上街去」初始提供六种可种植的植物。此后,每累计一万有效步数,人就能从其余植物中自由选择一种,加入植物库。步数可以跨天积累,解锁后的植物可以反复种植。

Plant choices are connected to everyday walking. Shangjiequ initially offers six plants. For every 10,000 valid steps accumulated after that, a person can freely choose one of the remaining plants to add to their collection. Steps can accumulate across days, and unlocked plants can be planted again and again.

目前,产品已支持由世界模型根据地点照片生成三维场景,并从地图回访。我计划独立保存植物及其位置,并关联地点,让人回访时再次看到当时种下的植物。

The product already supports generating 3D scenes from photos of a place using a world model, and revisiting those scenes from the map. I plan to store plants and their positions separately and link them to places, so that people can see what they planted when they return.

这就是我重新定义的 Human Movement ,我认为意义存在于空白与间隙处。

This is my redefinition of Human Movement. I believe meaning exists in the empty spaces and intervals.

TRY CARRY THE COSMOS / 体验上街去

在「上街去」,体验这一刻。Experience this moment in Carry the Cosmos.

在支持 AR 的 iPhone 上通过 TestFlight 打开「上街去」,点击左侧「AR 种植」直接进入相机,允许相机访问,再选择种植或伸手遇鸟。网页版用于浏览产品界面、内容与入口。Open Carry the Cosmos through TestFlight on an AR-capable iPhone. Tap AR Plant on the left to enter the camera, allow camera access, then choose planting or the bird encounter. The web version previews the interface, content and entry points.

本次方案基于「上街去」展开,聚焦一个停留时刻:身体行动如何获得回应。图册、AR 说明书及上方网页和 iPhone 入口均对应「上街去」。其最新本地实现中,种植与遇鸟共用相机入口,植物记录、地图位置及同机 AR 现场复访已实现;将同一株植物融合进照片生成的三维场景仍在规划。This case is developed within Carry the Cosmos and focuses on one pause: how bodily action receives a response. The plates, AR guide, web link and iPhone link all refer to Carry the Cosmos. In its latest local implementation, planting and the bird encounter share a camera entry, with plant records, map locations and same-device AR revisits implemented. Integrating the same plant into a photo-generated 3D scene remains planned.