Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-3859] [table] Add BigDecimal/BigInteger support to Table API #2088

Closed

Conversation

twalthr
Copy link
Contributor

@twalthr twalthr commented Jun 9, 2016

  • General
    • The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
    • The pull request addresses only one issue
    • Each commit in the PR has a meaningful commit message (including the JIRA id)
  • Documentation
    • Documentation has been added for new functionality
    • Old documentation affected by the pull request has been updated
    • JavaDoc for public methods has been added
  • Tests & Build
    • Functionality added by the pull request is covered by tests
    • mvn clean verify has been executed successfully locally or a Travis build has passed

This PR introduces the BigDecimal type (and thus also BigInteger) to the Table API. Basic arithmetic operations as well as ABS, FLOOR and CEIL are now possible. I fixed several bugs and refactored some code parts (e.g. I introduced an ExpressionTestBase).

@twalthr twalthr force-pushed the TableApiBigDecBigIntIntegration branch 2 times, most recently from d0eee42 to 812febc Compare June 10, 2016 11:39
s"'${genExpr.resultType}'")
}

def requireNumericOrComparable(genExpr: GeneratedExpression) =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename to requireComparable()?

@fhueske
Copy link
Contributor

fhueske commented Jun 13, 2016

Thanks for the PR, @twalthr! Looks mostly good. I found a few cases that needs to be fix.

We also should think about how to handle SQL DECIMAL types with fixed precision / scale, e.g., how to handle something like SELECT myDouble AS DECIMAL(4,2). Do you think this could be easily added with this PR or rather be fix in a later issue?

@fhueske
Copy link
Contributor

fhueske commented Jun 13, 2016

Think we also need aggregation functions for DECIMAL otherwise this won't work:

SELECT sum(myInt * 1.23) FROM MyTable

@twalthr I can implement the aggregation functions, if you want to.

@twalthr
Copy link
Contributor Author

twalthr commented Jun 13, 2016

@fhueske Thanks for reviewing my code. I totally forgot to test the comparisions. I will fix the issues and see what I can do for fixed precision / scale cases. Would be great if you could implement the corresponding aggregation functions.

@fhueske
Copy link
Contributor

fhueske commented Jun 14, 2016

Hi @twalthr, I implemented the aggregation functions and pushed the commit to a branch in my repository: https://github.com/fhueske/flink/tree/tableDecimal
I tried to open a PR against your repository, but Github didn't offer it as an option. :-(

@twalthr twalthr force-pushed the TableApiBigDecBigIntIntegration branch 2 times, most recently from 352674d to 92d2ce8 Compare June 15, 2016 11:07
@twalthr
Copy link
Contributor Author

twalthr commented Jun 15, 2016

@fhueske I have updated my PR. Now comparisons should work. Once the build succeeded, I will add your commits.

@fhueske
Copy link
Contributor

fhueske commented Jun 15, 2016

Thanks for the update @twalthr. The changes look good. Should be good to merge after the aggregators commit is added.

@twalthr twalthr force-pushed the TableApiBigDecBigIntIntegration branch from 92d2ce8 to 44bdf26 Compare June 17, 2016 07:47
@fhueske
Copy link
Contributor

fhueske commented Jun 17, 2016

Merging

fhueske pushed a commit to fhueske/flink that referenced this pull request Jun 17, 2016
@fhueske
Copy link
Contributor

fhueske commented Jun 18, 2016

Won't merge this PR at this point. A few Travis builds timed out and I found that the added expression tests increase build time by about one minute due to the code-gen compilation overhead.

@twalthr, do you think we can adapt the tests to batch several expressions into a single class with multiple methods to invoke the compiler less frequently?

@twalthr
Copy link
Contributor Author

twalthr commented Jun 19, 2016

I also already recognized the problem of long running tests. I think will rework the test base again. So that there is one compilation per unit test.

@twalthr twalthr force-pushed the TableApiBigDecBigIntIntegration branch from 077fedb to ef4d5ca Compare June 21, 2016 15:54
@twalthr
Copy link
Contributor Author

twalthr commented Jun 21, 2016

@fhueske I have changed the ExpressionTestBase. Now one unit tests only needs one compilation step. The runtime is still not perfect but could be reduced by about 75 % (at least on my PC).

@twalthr
Copy link
Contributor Author

twalthr commented Jun 23, 2016

If there are no objections, I would merge this later today...

@fhueske
Copy link
Contributor

fhueske commented Jun 23, 2016

Thanks for refactoring the tests @twalthr! LGTM

@twalthr
Copy link
Contributor Author

twalthr commented Jun 23, 2016

Thanks Fabian. Merging...

@asfgit asfgit closed this in 37defbb Jun 23, 2016
hequn8128 pushed a commit to hequn8128/flink that referenced this pull request Jun 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants