TOP

MEMBERS

RECRUIT

PUBLICATIONS

BLOG

COMPANY

CONTACT

INTERVIEW

We gained insights into the inner workings of the Metaverse Lab as well as what aspects captivate the two researchers.

Profile

Tomohiro Hayase, Ph.D.

In 2019, Dr. Hayase completed his Ph.D. in Mathematical Sciences at the Graduate School of Mathematical Sciences at the University of Tokyo. His doctoral research concerned machine learning using random matrices, free probability theory, and operator algebras. During his time at the graduate school, he worked as an intern in machine learning at Morpho, Inc. and Preferred Networks (PFN), and tackled model compression of deep neural networks. He has also been playing on various social virtual reality platforms since his time at graduate school, with his play time totaling a few thousand hours. After graduating, he conducted research on theoretical research on deep neural networks, contrastive learning, and transfer learning at Fujitsu’s Artificial Intelligence Laboratory, while working as a part-time lecturer (Information Science) at Ochanomizu University. Discovering the appeal of researching the metaverse while working as a world creator in social virtual realities, he joined Cluster Metaverse Lab as the first researcher in 2022.

Takefumi Hiraki, Ph.D.

Dr. Hiraki received his B.S., M.S., and Ph.D. degrees from the University of Tokyo, Japan in 2014, 2016, and 2019, respectively. From 2019, he worked as a Japan Society for the Promotion of Science (JSPS) Research Fellow at the Graduate School of Engineering Science, Department of Systems Innovation at Osaka University, and became a visiting researcher at Microsoft Research Asia in China. Since 2021, he served as an assistant professor at the Faculty of Library, Information and Media Science at the University of Tsukuba. Currently, he is a senior research scientist at Cluster Metaverse Lab at Cluster, Inc. while simultaneously holding an assistant professor position at the University of Tsukuba using the cross-appointment system. His research interests include augmented reality, haptic displays, soft robotics, and human-computer interaction.

What is the Metaverse Lab?

The Metaverse Lab leads Cluster's main goal to "accelerate human creativity". We conduct research in the fields of computer vision (CV), computer graphics (CG), human-computer interaction (HCI), virtual reality (VR), and brain machine interface (BMI) as well as cross-cutting research in machine learning (ML) using scientific knowledge and data accumulated in our platform. Our aim is to produce results that can be returned to the platform "cluster" in the short and long term, and to spur academic research to promote the progress of humanity as a whole, as well as to integrate them.

First, can you two briefly tell us about your background?

Hayase

I was at the Graduate School of Mathematical Sciences researching things like applied research on infinite-dimensional matrices. While taking part in the doctorate program, I did a CV/ML internship and after graduating, was part of the Artificial Intelligence Laboratory at Fujitsu researching CV and ML cameras. You might be wondering what the connection is to VR; I was interested in VR and played around with it during my time at university. I did collaborative research with Hiraki and played VRChat and cluster as a hobby, becoming a heavy user. I thought combining this with machine learning would be fun, and coincidentally, Cluster had created a research institute fit for this purpose, so I joined.

Hiraki

In graduate school, I was part of the Naemura Laboratory which researches fields of VR, AR, and MR, and I researched AR systems using high-speed projectors. Also, I conducted research with the JST ERATO Kawahara Universal Information Network Project, working with soft robotics which are robots created with soft materials, and the application of tactile displays on these robots. On a side note, a big part of my interest in VR came from anime. Generally speaking, “Sword Art Online” is thought of as the entrance to VR, but in my case, animes such as “.hack” and “serial experiments lain” sparked my interest in VR.

Hayase

Oh, if we’re talking about that, “Digimon Adventure: Our War Game!” was it for me.

Hiraki

Ahh, yeah, that too!

Hiraki

I had an interest in VR but I also loved making robots since middle school. I joined the robotics club in middle and high school, and joined RoboTech, a robotics competition club at the University of Tokyo. I initially planned to do research on robots given my background, but after reading an article on a VR system by Dr. Narumi[1] dealing with taste, Meta Cookie[2], during my first year, I reconfirmed my interest in VR and became a researcher in VR, AR, and MR.

[1] Associate Professor Takuji Narumi: Majored in Mechano-Informatics at the Graduate School of Information Science and Technology, University of Tokyo. Affiliated with the Kuzuoka, Amemiya, and Narumi Laboratories

[2] Meta Cookie: A system influencing the cognition of taste, changing the ‘flavor’ of a cookie for the person consuming it by superimposing visual and olfactory information of cookies

Thank you. What type of research is actually being conducted in the Metaverse Lab?

Hayase

The Metaverse Lab directly undertakes the mission of Cluster, to "accelerate human creativity". Firstly, to increase the possibilities of humanity, we build supportive AI and collective intelligence and accumulate data to make technologies that can do what human beings can. From the get-go, the good thing about machine learning is that you’re able to do multiple tasks where previously, you were only able to do only one. Drawing art, building something… The field of the Metaverse Lab is divided into others such as ML, VR/HCI, and BMI, but I believe the final objective of all these areas is connected to opening up possibilities for human beings and accelerating human creativity. While cluster provides the space, the lab handles the scientific aspects, working with algorithms and utilizing the power of humanity.

Hiraki

I see eye to eye with Hayase in that, I also believe the Metaverse Lab is directly involved with Cluster’s mission. With that in mind, I think the Metaverse Lab has two responsibilities: First, research in the field of academia. In other words, though not necessarily useful for the company, a form of research that makes advancements in science and technology for society as a whole. To accelerate the overall creativity of humanity, making advancements in science and technology through publications is a vital role of this institution. Second, research unique to Cluster, can be utilized by the company. That is, research to accelerate knowledge production in the field of VR, starting with cluster. For example, taking advantage of the large user base in cluster, we are thinking of anonymizing the compiled data and analyzing user behavior. We are also considering using the VR environment to conduct experiments and produce results that can be implemented to improve the platform. Here, it is important to keep in mind that we also need practical, implementable outputs, not only academic journal publications on innovative technologies.

Senior Research Scientist Tomohiro Hayase, Ph.D.

I see, that sounds interesting! On that note, why did you two decide to join the Metaverse Lab?

Hayase

I joined in the hopes of working with VR, ML, and Math altogether. With ML, it's always better to research using a platform because there is plenty of data. But I wanted to work with social VR in particular, because it was something I used day-to-day and something I felt the inconveniences of as well. I’ve always wished that the virtual world would be even more appealing to everyone - and it was then that Cluster founded the Metaverse Lab, and I thought it was the perfect opportunity to study ML here.

Hiraki

I decided to join the Metaverse Lab because I was drawn to the prospect of VR research in a private Japanese firm, and I thought that I would be able to conduct research suited for practical application by using the metaverse platform and the accumulated data in cluster. This is a first step towards promoting industry-based research in the field of VR and encouraging more interaction between academia and professional research, and I would like to contribute to this new trend led by the Metaverse Lab.

Senior Research Scientist Takefumi Hiraki, Ph.D.

Thank you very much. Then, what about the Metaverse Lab is particularly interesting for you two?

Hiraki

I found it fascinating that everyone involved in the Metaverse Lab was serious about creating a new research institution. Even though the lab can access corporate resources, it ensures autonomy and flexibility by obtaining a research institute number.[3] Plus, the company not only encourages research in VR/AR/MR but also runs its own metaverse platform which is the largest in the country. I think it's really exciting to be in an environment like this where we get to collaborate with the engineering team that is developing the platform as we speak. Another reason I felt compelled to join was when Mr. Kato, the CEO, expressed his enthusiasm about creating a research organization that can aim to compete on a global scale, rather than simply an R&D department in a firm.

[3] Research Institute Number: Granted by the Minister of Education, Culture, Sports, Science and Technology to designated “research institutions”, which allows it to apply for competitive research funding, such as the Grant-in-Aid for Scientific Research (KAKEN).

Hayase

I agree. I wouldn’t have joined either if it was simply an R&D department, but I was convinced that it was going to become a full-out research institution. For a typical R&D department, it’s enough to act as an intermediary between the company and the system or operations it is working on. But as a research institution, it would have to produce research outputs that are on par with global standards of research, on top of feeding back into the company. I believe that a research institution can only be considered a solid one when both these elements are in place.

Hiraki

In that sense, the CEO never asked us to research using cluster. It’s more like we, the researchers come up with research topics involving cluster because it would be a waste not to use it as we work here, and there’s a metaverse platform waiting to be researched. To be honest, I think researchers hate being what to research the most (laughs).

Hayase

You’re right about that one (laughs).

Hiraki

I am a researcher because I value freedom. That’s why I really appreciated how CEO Kato approached me by saying that he wasn’t looking for specific research; just research papers that hold value for humanity and society, and helps it accelerate.

Hayase

The importance of data and information is beginning to be recognized in the sciences. For instance, based on the idea that data itself can influence the physical world, quantum and thermodynamics are being integrated with information science. At the end of the day, information connects the world, the people, and science, and I believe the field of VR plays a role in this development. It’s exciting to think that if we fuse data, information, and algorithms (and hardware if possible), we’ll begin to see an interesting world, which in the end would be useful for humanity.

Hiraki

In the anime “PSYCHO-PASS”, they use a technology called holograms, which can be used to freely change the physical properties of objects, such as appearance and texture. I’d like to create technology like that in the future. You know, then we’ll be able to rewrite entire surfaces in a matter of seconds, and I think this would enrich the world we live in (laughs).

Hayase

(Laughs).

That would be very exciting!
I assume you’re currently recruiting team members to join you in the Metaverse Lab; what sort of people would you like to see join?

Hayase

Hmm, yes. I’d be happy to see people who simply find our conversation so far exciting and can relate (laughs).

Hiraki

If you apply for the position with the feeling that it is an R&D-like organization of a general venture company, I think there is a possibility of a mismatch. Like academia, we’d like you to see us as a serious research institute that basically has people who have completed doctoral courses, publishes papers at international conferences and in anglophone journals, and obtains competitive funds such as Grants-in-Aid for Scientific Research (KAKENHI) when necessary. I think there is a gap between a so-called "general R&D organization" and the organization we are aiming for, so I look forward to having team members who are serious about research or who already have a career as a researcher.

Hayase

Oh, and I’d like to add that there is no need to specialize in the same fields as us. If you can pursue research in an area you’re interested in, with curiosity and motivation, that’ll be perfect.

Hiraki

There is no top-down pressure at the Metaverse Lab, and researchers have a high degree of freedom. Plus, you’ll be able to engage in discussions with other researchers, who are also highly knowledgeable in their fields. It’s really a great environment for people looking to freely discuss matters and be inspired by fellow researchers. We look forward to welcoming new members into our midst.