<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>BERT on Subculture Science Research Group&#39;s Blog</title>
    <link>https://www.subcul-science.com/en/tags/bert/</link>
    <description>Recent content in BERT on Subculture Science Research Group&#39;s Blog</description>
    <generator>Hugo</generator>
    <language>en-US</language>
    <lastBuildDate>Wed, 03 Feb 2021 15:58:41 +0900</lastBuildDate>
    <atom:link href="https://www.subcul-science.com/en/tags/bert/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>How to train a Japanese model with Sentence transformer to get a distributed representation of a sentence</title>
      <link>https://www.subcul-science.com/en/post/20210203sbert/</link>
      <pubDate>Wed, 03 Feb 2021 15:58:41 +0900</pubDate>
      <guid>https://www.subcul-science.com/en/post/20210203sbert/</guid>
      <description>&lt;p&gt;.&#xA;BERT is a model that can be powerfully applied to natural language processing tasks.&lt;/p&gt;&#xA;&lt;p&gt;However, it does not do a good job of capturing sentence-wise features.&lt;/p&gt;&#xA;&lt;p&gt;Some claim that sentence features appear in [ CLS\ ], but&#xA;This paper](&lt;a href=&#34;https://arxiv.org/abs/1908.10084&#34;&gt;https://arxiv.org/abs/1908.10084&lt;/a&gt;) claims that it does not contain that much useful information for the task.&lt;/p&gt;&#xA;&lt;p&gt;Sentence BERT is a model that extends BERT to be able to obtain features per sentence.&lt;/p&gt;&#xA;&lt;p&gt;The following are the steps to create Sentence BERT in Japanese.&lt;/p&gt;</description>
    </item>
    <item>
      <title>A note on how to use BERT learned from Japanese Wikipedia, now available</title>
      <link>https://www.subcul-science.com/en/post/wikipediabert/</link>
      <pubDate>Wed, 17 Jun 2020 07:34:00 +0900</pubDate>
      <guid>https://www.subcul-science.com/en/post/wikipediabert/</guid>
      <description>&lt;p&gt;huggingface has released a Japanese model for BERT.&lt;/p&gt;&#xA;&lt;p&gt;The Japanese model is included in transformers.&lt;/p&gt;&#xA;&lt;p&gt;However, I stumbled over a few things before I could get it to actually work in a Mac environment, so I&amp;rsquo;ll leave a note.&lt;/p&gt;&#xA;&lt;h2 id=&#34;preliminaries-installing-mecab&#34;&gt;Preliminaries: Installing mecab&lt;/h2&gt;&#xA;&lt;p&gt;The morphological analysis engine, mecab, is required to use BERT&amp;rsquo;s Japanese model.&lt;/p&gt;&#xA;&lt;p&gt;The tokenizer will probably ask for mecab.&lt;/p&gt;&#xA;&lt;p&gt;This time, we will use homebrew to install Mecab and ipadic.&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
