Re: [DISCUSS] Drop Python 2 support for 1.10

[Dian Fu]

Thanks everyone for your reply.  

So far all the reply tend to option 1 (dropping Python 2 support in 1.10) and will continue to hear if there are any other opinions.

@Jincheng @Hequn, you are right, things become more complicate if dropping Python 2 support is performed after Python UDF has been supported. Users will have to migrate their Python UDFs if they have used features which only are supported in Python 2.

Thanks @Yu for your suggestion. It makes much sense to me and will do that. Also CC @user and @user-zh ML in case any users are concerned about this.

Thanks,
Dian

> 在 2019年10月9日，下午1:14，Yu Li <[hidden email]> 写道：
>
> Thanks for bringing this up Dian.
>
> Since python 2.7 support was added in 1.9.0 and would be EOL near the
> planned release time for 1.10, I could see a good reason to take option 1.
>
> Please remember to add an explicit release note and would be better to send
> a notification in user ML about the plan to drop it, just in case some
> 1.9.0 users are already using python 2.7 in their product env.
>
> Best Regards,
> Yu
>
>
> On Wed, 9 Oct 2019 at 11:13, Jeff Zhang <[hidden email]> wrote:
>
>> +1
>>
>> Hequn Cheng <[hidden email]> 于2019年10月9日周三 上午11:07写道：
>>
>>> Hi Dian,
>>>
>>> +1 to drop Python 2 directly.
>>>
>>> Just as @jincheng said, things would be more complicated if we are going
>> to
>>> support python UDFs.
>>> The python UDFs will introduce a lot of python dependencies which will
>> also
>>> drop the support of Python 2, such as beam, pandas, pyarrow, etc.
>>> Given this and Python 2 will reach EOL on Jan 1 2020. I think we can drop
>>> Python 2 in Flink as well.
>>>
>>> As for the two options, I think we can drop it directly in 1.10. The
>>> flink-python is introduced just from 1.9, I think it's safe to drop it
>> for
>>> now.
>>> And we can also benefit from it when we add support for python UDFs.
>>>
>>> Best, Hequn
>>>
>>>
>>> On Wed, Oct 9, 2019 at 8:40 AM jincheng sun <[hidden email]>
>>> wrote:
>>>
>>>> Hi Dian，
>>>>
>>>> Thanks for bringing this discussion!
>>>>
>>>> In Flink 1.9 we only add Python Table API mapping to Java Table
>>> API(without
>>>> Python UDFs), there no special requirements for Python version, so we
>> add
>>>> python 2,7 support. But for Flink 1.10, we add the Python UDFs support,
>>>> i.e., user will add more python code in Flink job and more requirements
>>> for
>>>> the features of the Python language.So I think It's better to follow
>> the
>>>> rhythm of Python official.
>>>>
>>>> Option 2 is the most conservative and correct approach, but for the
>>> current
>>>> situation, we cooperate with the Beam community and use Beam's
>>> portability
>>>> framework for UDFs support, so we prefer to adopt the Option 1.
>>>>
>>>> Best,
>>>> Jincheng
>>>>
>>>>
>>>>
>>>> Dian Fu <[hidden email]> 于2019年10月8日周二 下午10:34写道：
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I would like to propose to drop Python 2 support(Currently Python
>> 2.7,
>>>>> 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at
>>> Jan
>>>>> 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are
>>>> planning
>>>>> to drop Python 2 support.
>>>>>
>>>>> The benefits of dropping Python 2 support are:
>>>>> 1. Maintaining Python 2/3 compatibility is a burden and it makes the
>>> code
>>>>> complicate as Python 2 and Python 3 is not compatible.
>>>>> 2. There are many features which are only available in Python 3.x
>> such
>>> as
>>>>> Type Hints[5]. We can only make use of this kind of features after
>>>> dropping
>>>>> the Python 2 support.
>>>>> 3. Flink-python depends on third-part projects, such as Apache Beam
>>> (may
>>>>> add more dependencies such as pandas, etc in the near future), it's
>> not
>>>>> possible to upgrade them to the latest version once they drop the
>>> Python
>>>> 2
>>>>> support.
>>>>>
>>>>> Here are the options we have:
>>>>> 1. Drop Python 2 support in 1.10:
>>>>> As flink-python module is a new module added since 1.9.0 and so
>>> dropping
>>>>> Python 2 support at the early stage seems a good choice for us.
>>>>> 2. Deprecate Python 2 in 1.10 and drop its support in 1.11:
>>>>> As 1.10 is planned to be released around the beginning of 2020. This
>> is
>>>>> also aligned with the official Python 2 support.
>>>>>
>>>>> Personally I prefer option 1 as flink-python is new module and there
>> is
>>>> no
>>>>> much history reasons to consider.
>>>>>
>>>>> Looking forward to your feedback!
>>>>>
>>>>> Regards,
>>>>> Dian
>>>>>
>>>>> [1] https://pythonclock.org/ <https://pythonclock.org/>
>>>>> [2] https://python3statement.org/ <https://python3statement.org/>
>>>>> [3]
>>>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
>>>>> <
>> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
>>>>
>>>>> [4]
>>>>>
>>>>
>>>
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>>>>> <
>>>>>
>>>>
>>>
>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
>>>>>>
>>>>> [5]
>>>>>
>>>>
>>>
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>>>>> <
>>>>>
>>>>
>>>
>> https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5
>>>>>>
>>>>
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>