Polycomb group (PcG) proteins play important roles in repressing lineage-specific genes and maintaining the undifferentiated state of mouse embryonic stem cells (mESCs). However, how PcG proteins are recruited to their target genes is largely unknown. Here, we show that the H3K36-specific histone demethylase Kdm2b is highly expressed in mESCs and regulated by the pluripotent factors Oct4 and Sox2 directly. Depletion of Kdm2b in mESCs causes de-repression of lineage-specific genes and induces early differentiation. The function of Kdm2b depends on its CxxC-ZF domain, which mediates its genome-wide binding to CpG islands (CGIs). Kdm2b interacts with the core components of polycomb repressive complex 1 (PRC1) and recruits the complex to the CGIs of early lineage-specific genes. Thus, our study not only reveals an Oct4–Sox2–Kdm2b–PRC1–CGI regulatory axis and its function in maintaining the undifferentiated state of mESCs, but also demonstrates a critical function of Kdm2b in recruiting PRC1 to the CGIs of lineage-specific genes to repress their expression.